Test Case Data Format

Alyvix test cases are stored as JSON files, which are then used by all Alyvix applications. Alyvix files represent the unique mechanism for interchange between Alyvix modules, and each file is self-contained, without any external dependencies.

Test Case Files

Each time you exit Designer or Selector by pressing the OK button, or save an object in Editor, the JSON object representing the test case is saved in the current directory under the file name:

<filename>.alyvix

where filename is the value of the -f parameter passed when starting Editor or Designer.

If you then call Alyvix Robot via the command line with that file as a parameter, when it finishes it will create a new file based on the original test case, but with the resulting measures included. The file name will have the timestamp at the moment of execution appended:

<filename>_<full-timestamp>.alyvix

For example:

logintest_20191220_145228_UTC+0100.alyvix

Note

This file is not produced when a test case is executed in Editor via runblue, only when executed directly from Robot.

Test Case JSON Structure

Alyvix Test Case Object JSON Structure

A single .alyvix file stores the entire test case and holds all measures resulting from a single execution run.

The following example JSON structure illustrates the high-level structure of the test case object:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
{
   "maps": { },
   "objects": {
      "<test-case-object-name>": {
         "call": { },
         "components": {
            "<resolution>": {
               "groups": [
                  { "main": { },
                    "subs": [ ] },
                  { "main": { },
                    "subs": [ ] },
                  { "main": { },
                    "subs": [ ] }
               ],
               "screen": "<base64>"
            }
         },
         "date_modified": "<timestamp>",
         "detection": {
            "break": true,
            "timeout_s": 10,
            "type": "appear" },
         "measure": {
            "output": true,
            "series": { },
            "thresholds": { } }
      }
   },
   "script": {
      "case": [
         "<test-case-object-name>",
         { "disable": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "if-true": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "if-false": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "for": "<test-case-map-name>" }
      ],
      "sections": {
         "<section-name>": [
            "<test-case-object-name>"
         ],
         "exit": [],
         "fail": [] }
   }
}

The individual sections of the JSON structure are explained below.

  1. The top level corresponds to the elements used in Alyvix Editor:

    { "maps": {
         "map-name":  { } },
      "objects": {
         "<test-case-object-name>":  { } },
      "script": {
         "case":  [ ],
         "sections":  { } }
    }
    
    1. maps An ordered set of values that a script can loop over, for instance to insert a sequence of values in multiple text fields

    2. objects A list of the individual test case objects created with Alyvix Designer. Each test case object is identified uniquely in the list by its object name .

    3. script The scripts created for a test case, both the main script and any section (subroutine) scripts.

  1. Maps

    Each named map under the maps field contains a set of named keys (rows) and a fixed number of cell values, to represent a m x n matrix (columns cannot be named):

    { "<map-name-1>": {
         "key1": [ "key1-val1",
                   "key1-val2" ],
         "key2": [ "key2-val1",
                   "key2-val2" ] }
    }
    
  1. Objects

    The value of each property contained within the objects substructure (shown here as “test-case-object-name”) corresponds to a test case object created in Designer. Those values have the following meanings:

    { "<test-case-object-name>": {
         "call": { },
         "components": { },
         "date_modified": "<timestamp>",
         "detection": { },
         "measure": {
             "series": { } }
      }
    }
    
    1. call The recorded options to start or kill an external application when a test case object is first called

    2. components The representation for the component tree (detailed below)

    3. date_modified The time the test case object was last modified, displayed in Selector

    4. detection The test case object’s detection conditions such as timeout, break, and appear/disappear

  1. Objects >> Components

    The components section represents the component tree of a test case object. The example below shows how you can have one component tree at each distinct screen resolution.

    { "components": {
         "<resolution>": {
            "groups": [
               { "main": {
                    "detection": { },
                    "interactions": { },
                    "visuals": { } },
                 "subs": [ ] },
               { "main": { },
                 "subs": [ ] },
               { "main": { },
                 "subs": [ ] } ],
            "screen": "<base64>" } }
    }
    
    1. <resolution> The resolution of a screen capture as shown in Alyvix Selector. This value is not selected by the user, but is dependent on the hardware and the settings in use when the screen is captured.

      1. groups A JSON array of exactly 3 items, each of which corresponds to one of the groups in the component tree.

        1. main The main component in a group.

          1. detection Information about the match method to use when trying to detect the region of interest.

          2. interactions Details on any mouse or keyboard actions to initiate in the event of a positive detection.

          3. visuals Information about the region of interest and the selection box laid over the screen capture.

        2. subs A JSON array of exactly 4 items, each of which corresponds to one of the subcomponents in a group. The structure of each element is identical to the structure of the main component above.

      2. screen The Base64 representation of the screen capture. There is exactly one screen capture for each resolution in a test case object.

  1. Objects >> Measure

    The measure section contains the recorded data of all runs by Alyvix Robot.

    { "measure": {
        "output": "<true/false>",
        "series": [
            {
                "accuracy_ms": "<ms_amount>",
                "annotation": "<base64>",
                "exit": "<true/false/fail/not_executed>",
                "performance_ms": "<ms_amount>",
                "records": {
                    "check": "<true/false>",
                    "extract": "<extracted_text>",
                    "image": "<extracted_image>",
                    "text": "<scraped_text>"
                },
                "resolution": {
                    "height": "<pixel_amount>",
                    "width": "<pixel_amount>"
                },
                "scaling_factor": "<zoom_amount>",
                "screenshot": "<base64>",
                "timestamp": "<epoch>"
            }
        ],
        "thresholds": { } }
    }
    
  1. Script

    The script parameter records the main script and any subroutines used to compose more complex interaction behaviors.

    { "case": [
         "<test-case-object-name>",
         { "disable": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "if-true": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "if-false": "<test-case-object-name>" },
         { "flow": "<test-case-object-name>",
           "for": "<test-case-map-name>" }
      ],
      "sections": {
         "<section-name>": [
            "<test-case-object-name>",
            "<test-case-object-name>"
         ],
         "exit": [],
         "fail": [] }
    }
    
    1. case The main script that will be executed by Alyvix Robot. The script is an ordered list of scripting elements, where a simple, enabled element is just the name of a test case object. Other elements have the following meanings:

      1. disable Skip this test case object and continue with the next.

      2. flow Indicates the test case object or section that is the value of this key should be executed by Alyvix Robot in the context of a conditional (if-true or if-false) or a loop (map) .

      3. if-true Evaluates whether the detection part of a given test case object would match, but without executing its actions.

      4. if-false As above, but if the test case object would not match.

      5. for Indicates there should be a loop over the values in the specified Map.

    2. sections Corresponds to the named subroutine scripts created in Alyvix Editor. Each section is a parameter where its key is the name of the section and its value is the same as the case value described above.

      1. exit The (teardown) script to execute when a script has completed, regardless of whether it succeeded or failed.

      2. fail The (teardown) script to execute when a script has failed during the execution of its test case objects.