Test Runner

This document describes Test Runner configuration file and command line parameters.

LINE_DELAY

Sets the amount of time in seconds before the next line of a script will be executed

Parameter	Description	Required
Delay	Delay in seconds before the next line is executed. A value of 0 means to execute the scripts as fast as possible.	True

PAUSE_ON_ERROR

Set or clear the pause on error checkbox

If this is checked, Test Runner will pause if the test encounters an error. Otherwise the error will be logged but the script will continue.

Parameter	Description	Required
Enable	Whether to pause when the script encounters an error Valid Values: TRUE, FALSE	True

MONITOR_LIMITS

Log limits events to the Script Runner log file while a script is running

Limits events are always logged by the Command and Telemetry Server but are not put in the Script Runner log without this keyword.

PAUSE_ON_RED

Pause a running script if a red limit occurs

LOAD_UTILITY

Specify a test procedure to load into Test Runner

Procedures will be found automatically in the procedures directory or can be given by a path relative to the COSMOS install directory or by an absolute path.

Parameter	Description	Required
Filename	Name of the test file in quotes	True

RESULTS_WRITER

Specify a different Ruby file to interpret and print the Test Runner results

The specified Ruby file must define a class which implements the Cosmos::ResultsWriter API

Parameter	Description	Required
Filename	Name of the Ruby file which implements a result writer	True
Class Parameters	Parameters to pass to the constructor of the results writer	False

ALLOW_DEBUG

Whether to allow the user to enable the debug line where the user can enter arbitrary statements

CONTINUE_TEST_CASE_AFTER_ERROR

Set or clear the continue test case after error checkbox

If this is checked, Test Runner will continue executing the current test case after encountering an error. Otherwise the test case will stop at the error and the next test case will execute.

Parameter	Description	Required
Enable	Whether to continue the test case when the script encounters an error Valid Values: TRUE, FALSE	True

ABORT_TESTING_AFTER_ERROR

Set or clear the abort testing after error checkbox

If this is checked, Test Runner will stop executing after the current test case completes (how it completes depends on CONTINUE_TEST_CASE_AFTER_ERROR). Otherwise the next test case will execute.

Parameter	Description	Required
Enable	Whether to continue to the next test case when the script encounters an error Valid Values: TRUE, FALSE	True

MANUAL

Set the $manual global variable for all executing scripts

The $manual variable can be checked during tests to allow for fully automated tests if it is not set, or for user input if it is set. This capability is completely dependent on user specific code which checks the $manual variable.

Parameter	Description	Required
Enable	Whether to set the $manual global to true Valid Values: TRUE, FALSE	True

LOOP_TESTING

Set or clear the loop testing checkbox

If this is checked, Test Runner will continue to run whatever level of tests that were initially started. If either “Abort Testing after Error” or “Break Loop after Error” are checked, then the loop testing will stop if an error is encountered. The difference is that the “Abort Testing after Error” will stop testing immediately after the current test case completes. “Break Loop after Error” continues the current loop by executing the remaining suite or group before stopping. In the case of executing a single test case the options effectively do the same thing.

Parameter	Description	Required
Enable	Whether to loop the selected test level Valid Values: TRUE, FALSE	True

BREAK_LOOP_AFTER_ERROR

Set or clear the break loop after error checkbox

If this is checked, Test Runner continues the current loop by executing the remaining suite or group before stopping.

Parameter	Description	Required
Enable	Whether to break the loop after encountering an error Valid Values: TRUE, FALSE	True

IGNORE_TEST

Ignore the given test class name when parsing the tests

Parameter	Description	Required
Test Class Name	The test class to ignore when building the list of available tests	True

IGNORE_TEST_SUITE

Ignores the given test suite name when parsing the tests

Parameter	Description	Required
Test Suite Name	The test suite to ignore when building the list of available tests	True

CREATE_DATA_PACKAGE

Creates a data package of every file created during the test

AUTO_CYCLE_LOGS

Automatically start a new server message log and cmd/tlm logs at the beginning and end of each test. Typically used in combination with CREATE_DATA_PACKAGE.

COLLECT_METADATA

Prompt for Meta Data before starting tests

DISABLE_TEST_SUITE_START

Disable the Test Suite Start button to prevent all tests in the Suite from running. This is a useful option if you’re simply using Suites and Groups to organize test cases but do not want a user to accidentally run them all.

DISABLE_TEST_GROUP_START

Disable the Test Group Start button to prevent all tests in the Group from running. This is a useful option if you’re simply using Suites and Groups to organize test cases but do not want a user to accidentally run them all.

Example File

Example File: <Cosmos::USERPATH>/config/tools/test_runner/test_runner.txt

REQUIRE_UTILITY example_test
ALLOW_DEBUG
PAUSE_ON_ERROR TRUE
CONTINUE_TEST_CASE_AFTER_ERROR TRUE
ABORT_TESTING_AFTER_ERROR FALSE
MANUAL TRUE
LOOP_TESTING TRUE
BREAK_LOOP_AFTER_ERROR TRUE
IGNORE_TEST ExampleTest
IGNORE_TEST_SUITE ExampleTestSuite

CREATE_DATA_PACKAGE
COLLECT_META_DATA META DATA

LINE_DELAY 0
MONITOR_LIMITS
PAUSE_ON_RED

Command Line Parameters

Usage: ruby TestRunner [options]

    -h, --help                       Show this message
    -v, --version                    Show version
        --system FILE                Use an alternative system.txt file
    -c, --config FILE                Use the specified configuration file
        --stylesheet FILE            Use the specified stylesheet

Window Size Options:
        --minimized                  Start the tool minimized
        --maximized                  Start the tool maximized
        --defaultsize                Start the tool in its default size
        --stay-on-top                Force the tool to stay on top of all other windows

Window X & Y Position Options:
  Positive values indicate a position from the top and left of the screen.
  Negative values indicate a position from the bottom and right of the screen.
  A value of -1 indicates to place the right or bottom side of the window
  next to the right or bottom edge of the screen.
    -x, --xpos VALUE                 Window X position
    -y, --ypos VALUE                 Window Y position

Window Width and Height Options:
  Specifing width and height will force the specified dimension.
  Otherwise the window will layout according to its defaults.
    -w, --width VALUE                Window width
    -t, --height VALUE               Window height

Test Runner Specific Options:
    -s, --server FILE                Use the specified server configuration file for disconnect mode
        --suite SUITE                Start the specified test suite.
        --group GROUP                Start the specified test group. Requires the --suite option.
        --case CASE                  Start the specified test case. Requires the --suite and --group options.