Skip to content

Squash TM test execution plan retrieval with a workflow

Squash gives you the possibility to insert, in a CI/CD pipeline, a step to retrieve and execute an automated test plan defined in Squash TM using a workflow written in YAML or JSON. In the case of Jenkins, the configuration of this step can be simplified with a plugin, see the corresponding page of this guide.
You can find more information regarding the redaction of a workflow in the OpenTestFactory Orchestrator documentation.

Prerequisites

A user belonging to the Test automation server group must exist in Squash TM.

Integration of the Squash TM execution plan retrieval step into a workflow

The execution plan, iteration or test suite, must contain at least one ITPI (Iteration Test Plan Item, see the Squash TM glossary) linked to an automated test case, as described in the instructions here.

In order to retrieve this execution plan from Squash TM with a workflow, you need to call the corresponding generator action.

Here is a simple example of such a workflow in JSON format:

{
    "apiVersion": "opentestfactory.org/v1alpha1",
    "kind": "Workflow",
    "metadata": {
        "name": "Simple Workflow"
    },
    "defaults": {
        "runs-on":"ssh"
    },
    "jobs": {
        "explicitJob": {
            "runs-on":"ssh",
            "generator":"tm.squashtest.org/tm.generator@v1",
            "with": {
                "testPlanUuid":"1e2ae123-6b67-44b2-b229-274ea17ad489",
                "testPlanType":"Iteration",
                "squashTMUrl":"https://squashtm.example.com/squash",
                "squashTMAutomatedServerLogin":"tfserver",
                "squashTMAutomatedServerPassword":"tfserver",
                "technologyLabels":{
                    "ranorex": ["windows","ranorex"],
                    "robotframework": ["linux","robotframework"],
                    "junit": ["linux","junit"]
                }
            }
        }
    }
}

Mandatory Fields

A Squash TM generator step must contain the following parameters:

  • squashTMUrl: Public URL of Squash TM.
  • testPlanType: Refers to the type of test plan to retrieve in Squash TM. Only Iteration and TestSuite values are accepted.
  • testPlanUuid: Refers to the UUID (Universally Unique Identifier, see the dedicated documentation of the desired test plan. This can be found in the Information block of the desired iteration or test suite in Squash TM.

For API token authentication:

  • squashTMAutomatedServerToken: API token of the user from the Test Automation Server group to be used in Squash TM;

For more information on creating an API token for a user in the Test Automation Server group, please refer to the dedicated documentation.

For basic authentication (login/password):

  • squashTMAutomatedServerLogin: Username of the user from the Test Automation Server group to be used in Squash TM.
  • squashTMAutomatedServerPassword: Password of the user from the Test Automation Server group to be used in Squash TM.

Optional Fields

  • tagLabel: Specific to the Ultimate version 💎 - It refers to the name of the tag type custom field on which the test cases to retrieve are to be filtered.
    It is not possible to specify more than one.
  • tagValue: Specific to the Ultimate version 💎 - It refers to the value of the tag type custom field on which the test cases to retrieve are to be filtered.
    It is possible to specify multiple ones separated by | (example value1|value2). There has to be at least one value specified for the test case to be taken into account.

Focus

If one of the two tagLabel or tagValue fields is present, the other one must also be specified.

  • technologyLabels: To use if you retrieve an execution plan containing tests which must be executed on different environments.
    It enables you to specify, for each automation framework, targeted environment execution's tags.
    They will be added to the ones specified in the job's runs-on attribute.
    For a technology, tags are defined by adding an entry to the field list with format "technology prefix": ["tag1", "tag2"].

Info

By default, jobs created by the Generator after retrieving an execution plan are tagged with the corresponding technology prefix (technology name in lower case with no space, for example "robotframework" for Robot Framework) in addition of the ones specified in Generator job's runs-on attribute.

Execution order of tests

The only order assured by Squash is that, for a given Git repository in a given iteration/test suite, the tests will be executed in the order defined in Squash TM.
If a test iteration/suite contains automated tests belonging to multiple Git repositories, the order of execution of tests from one repository relative to the execution of tests from another repository is undefined.
If several test iterations/suites are launched from Squash TM (resp. the CI/CD pipeline), the order of execution of the tests of one iteration/suite relative to the execution of the tests of another iteration/suite is undefined. (Except in the trivial case where a suitable execution environment is available and the first suite/iteration was able to start before the Squash TM user – resp. the CI/CD pipeline – launches another one.)

Usage of the orchestrator with inception execution environment

OpenTestFactory, and therefore Squash, have a special execution environment called inception to use an alternative use mode for generator type jobs.

What is the purpose of inception environment?

Usual execution process of a generator job is to retrieve an execution plan from Squash TM and then execute automated tests of this plan on dedicated execution environment. When an inception execution environment is used by a generator job, there is no test execution following execution plan retrieval. Instead, result publication step to Squash TM is run based on a set of reports sent upstream to the orchestrator.

How to use inception environment in a workflow

Below is a workflow example in YAML format using inception execution environment and retrieving a Squash TM execution plan.

metadata:
  name: Test Inception
resources:
  files:
  - report1
  - report2
  - report3
jobs:
  prepare:
    runs-on: inception
    steps:
    - uses: actions/prepare-inception@v1
      with:
        report.html: ${{ resources.files.report1 }}
        output.xml: ${{ resources.files.report2 }}
        log.html: ${{ resources.files.report3 }}
  robot:
    runs-on: inception
    needs: [prepare]
    generator: tm.squashtest.org/tm.generator@v1
    with:
      squashTMUrl: https://squashtm.example.com/squash
      squashTMAutomatedServerLogin: ${{ variables.SQUASH_USER }}
      squashTMAutomatedServerPassword: ${{ variables.SQUASH_PASSWORD }}
      testPlanUuid: 1e2ae123-6b67-44b2-b229-274ea17ad489
      testPlanType: Iteration

There are several differences compared to the workflow of the previous section:

  • The files section in the resources part: it lists the reports provided to the orchestrator. Information published to Squash TM will be extracted from these reports.

  • A preparation job, called prepare in this example: it uses actions/prepare-inception@v1 action provider and allows to indicate what reports, usually generated by the test execution, match with the reports indicated in the files section.
    The list of reports supported by Squash Orchestrator is indicated on the page corresponding to each test technology:

    The first report in the list (which is the Surefire report for most test technologies) must be present for the orchestrator to determine the test result.

  • The generator job: it is very similar to the one of the previous section with two differences:

    • It uses inception environment in the runs-on field
    • It specifies that the preparation job is a prerequisite for its execution with the line needs: [prepare]

Such a workflow can be run with the following command:

curl -X POST \
     -H "Authorization: Bearer ${TOKEN}" \
     -F workflow=@Squashfile \
     -F report1=@report1.html \
     -F report2=@report2.xml \
     -F report3=@report3.xml \
     -F variables="SQUASH_USER=${USER}\nSQUASH_PASSWORD=${PASSWD}" \
     http://orchestrator.example.com/workflows

curl -X POST ^
     -H "Authorization: Bearer %TOKEN%" ^
     -F workflow=@Squashfile ^
     -F report1=@report1.html ^
     -F report2=@report2.xml ^
     -F report3=@report3.xml ^
     -F variables="SQUASH_USER=%USER%\nSQUASH_PASSWORD=%PASSWD%" ^
     http://orchestrator.example.com/workflows

curl -X POST `
     -H "Authorization: Bearer $Env:TOKEN" `
     -F workflow=@Squashfile `
     -F report1=@report1.html `
     -F report2=@report2.xml `
     -F report3=@report3.xml `
     -F variables="SQUASH_USER=$Env:USER\nSQUASH_PASSWORD=$Env:PASSWD" `
     http://orchestrator.example.com/workflows

Note in this command the parameters specifying the reports to be sent by the orchestrator to Squash TM.

A workflow using inception execution environment can also be run from a Jenkins pipeline thanks to the Jenkins plugin step.

Squash TM parameters to exploit in an automated test

By executing a workflow retrieving a Squash TM execution plan, Squash TM passes various pieces of information on execution plan items that can be exploited during test case execution.

The following frameworks can exploit Squash TM parameters during test execution:

Each of these pages describes the parameters that can be retrieved by the test script and how to retrieve them.

Results publication in Squash TM at the end of the execution

The results and reports of the executed tests are transmitted and accessible in Squash TM in the same manner as for an execution initiated from it, refer to this paragraph.