Test Case Execution

Alyvix test cases you have built with Alyvix Editor can be run using Alyvix Robot. Robot will also execute any individual test case objects you have created with Alyvix Designer, along with test case objects you have duplicated with Alyvix Selector.

In a production environment the typical use case is to create a set of test cases once, and then repeatedly run those test cases continuously (ideally as quickly as possible). Examples include monitoring the usability of proprietary clients, streamed applications, and web pages on a remote server. A test case may measure responsiveness every 5 minutes, for instance, and send timing and other data to a monitoring system in the standardized output format it expects.

Before you can use Alyvix in a production environment, however, you will first need to iteratively develop and improve your test cases. Here human readable output is more important, and accordingly there is no output format option within Alyvix Editor.

There are two principal modes for interacting with Alyvix Robot to ensure test cases work properly during the development phase:

  • Calling Robot from within Alyvix Editor via the runblue button to run the main script

  • Calling Robot from the Command Prompt, passing it the name of a test case object, and using the default --mode option (alyvix/CLI)

In either case, one test case is executed at a time.


You can also run individual elements of a test case’s script which can be especially useful when debugging them.

Launching Alyvix Robot from the Command Prompt

When Robot is run in the command prompt, it first checks to see if one or more test case object names were passed via the -o parameter. If so, Robot will execute those test case objects in sequence. Otherwise, if the -o parameter is not specified, but the .alyvix test case file named by the -f parameter contains a script previously defined in Editor, then Robot will run that script.

Adding the verbosity (-v) parameter will provide additional information that can help you should you need to debug your test cases (see the section CLI Output Format below).

Alyvix Robot will execute an Alyvix file’s test case script when run with the following command (without the .alyvix suffix):

C:\Alyvix\testcases> alyvix_robot -f <alyvix-file-name>

If you used Alyvix Editor to create a file named start-test.alyvix containing a test case object named start, you can run just that test case object with this command:

C:\Alyvix\testcases> alyvix_robot -f start-test -o start

Robot also allows you to execute multiple test case objects in sequence by putting them in order in quotation marks after the --object parameter, as long as all those objects exist in the test case:

C:\Alyvix\testcases> alyvix_robot -f start-test -o "start settings"

The following options are available:








Supply one or more strings to use in the String field of a test case object in Designer




Supply the file name with or without extension




Supply a private key for use with encryption




alyvix — CLI output format for humans (default)

nagios — Nagios output (see below)

nats-influxdb — NATS to InfluxDB (see below)




Supply the Object name(s)




Set the verbosity level for debugging output ranging from 0 (min, default) to 2 (max)

0: Records start/stop timestamps, state and time measures for each object (with measure option enabled)

1: Also logs Alyvix actions

2: Also creates screenshots and annotated screenshots as separate .png files in the same directory

What Alyvix Robot Returns

Alyvix uses the following industry-standard return values for monitoring systems:

Error Level





The service responded and the results were within expectations



Action should be taken to prevent a likely near-term problem from becoming more serious



A significant incident has already occurred and should be handled immediately

Using the --mode option, you can specify the format for the information returned (defaults to CLI output mode).

CLI Output Format

When run from the command prompt with the default --mode parameter, Alyvix Robot will both (a) display a short log describing basic events and timing data, and (b) create a new file based on the original test case, but with more detailed time measures added. A timestamp corresponding to the moment of execution will be appended to the file name:


For example:


When run from the command prompt with the default -m alyvix parameter, Alyvix Robot will return human-readable output like the following when successful:

2019/12/12 18:24:20.405: start starts
2019/12/12 18:24:21.949: open DETECTED in 0.0s (+/-0.060)
2019/12/12 18:24:21.950: start ends OK, taking 1.544s.

If it fails instead, the output will appear like this:

2019/12/12 18:37:41.448: start-test starts
2019/12/12 18:37:51.762: settings FAILED after 10s
2019/12/12 18:37:51.763: start-test ends FAILED because of settings, taking 10.315s.

If you have enabled the break flag in Alyvix Selector for a given test case object, then if that test case object fails, no further test case objects will be executed. If it is not set, then the test case object will instead be skipped after its timeout has expired.

When run from the Windows command prompt, you can access the return value programmatically as follows:

C:\Alyvix\testcases> echo %errorlevel%

Multiple Measures in CLI Output Mode

If a given test case object is run more than one time in a single script, regardless of whether it appears in a sequential node, a conditional or inside a section, only the output of the object’s last run will be displayed. However, all results are still stored on the series element of the test case object. Should you need to see the output of a single test case object when it runs more than once, best practice is to use Selector to duplicate it (this will improve any data visualizations used with the data).

The exception is when a test case object appears in a loop. In this case, the timing results will be output for each key in the loop’s map. For instance, the following example might be output after running the script FORmy_mapRUNdataentry, using the pattern <object_name>_<map_name>-<key_name>:

2020/06/03 10:17:08.732: start-test starts
2020/06/03 10:17:10.993: dataentry_loop-key1 DETECTED in 0.0s (+/-0.062)
2020/06/03 10:17:13.370: dataentry_loop-key2 DETECTED in 0.0s (+/-0.060)
2020/06/03 10:17:13.372: start-test ends OK, taking 4.640s.

Nagios Output Format

When Alyvix Robot is run from the command prompt, the --mode nagios command option will generate performance data in Nagios message format . The main status result for the monitoring check will be one of the following values:




When the test case exits with OK


When the test case exits with OK, but the performance of at least 1 object is greater than its warning threshold and still lower than its critical threshold


When the test case exits with OK, but the performance of at least 1 object is greater than its critical threshold and still lower than its threshold threshold


Whenever the test case exits with FAILED

The time measurements for Nagios are specified as follows (note that object_timeout_s for SKIPPED must be converted to milliseconds):

Test Case Object Status










ms       (no value)



ms       (no value)


First nagios output line:

<test_case_output_status>: <test_case_output_message> | duration=<test_case_duration_s>s;;; <object_01_name>=<object_01_performance_ms>ms;<object_01_warning_s>s;<object_01_critical_s>s;; <object_02_name>=<object_02_performance_ms>ms;<object_02_warning_s>s;<object_02_critical_s>s;;

Second nagios output line:

FAILED transactions (from first to last): <failed_object_01_name>; <failed_object_02_name>

Third nagios output line:

NOT EXECUTED transactions: <not_executed_object_01_name>; <not_executed_object_02_name>

NATS-InfluxDB Output Format

This will publish all measures to an InfluxDB database through a NATS channel. You will need the NATS server IP, port, subject name, and measurement name.

C:\Alyvix\testcases> alyvix_robot -f vt -m "nats-influxdb <nats_streaming_server_ip>:<port> <influxdb_subject_name> <influxdb_measurement_name>"

Alyvix Cipher for Encryption

You can use a combination of cipher and private key to protect sensitive information such as user names and passwords when for instance you need to log in to a login-protected application or web site. The Videos and Tutorials section provides examples of how you can do this using Microsoft’s RDC and Citrix.

To encrypt a cipher, supply the text to be encrypted and your private key:

C:\Alyvix\testcases> alyvix_cipher -e <text_string_to_encrypt> -k <private_key>

You can also decrypt a cipher as follows:

C:\Alyvix\testcases> alyvix_cipher -d <text_string_to_decrypt> -k <private_key>

To use the encrypted text string, put it in the string box of an object component.

You can then run a test case with encrypted strings by supplying the private key:

C:\Alyvix\testcases> alyvix_robot -f <test_case_name> -k <private_key>