Execute automated test cases

Running automated tests

What it does

Automated tests are identified in the execution plan of an iteration or test suite by the icon displayed in the Mode column.

It is possible to execute all the automated test cases, included in the execution plan, or a selection of those, by clicking on the button at the top right of the execution plan:

It is also possible to execute a single automated test case by clicking on the Automated execution sub-menu:

When the automated execution starts, an Overview of the automated test execution pop-up is displayed and an automated suite is created.

Depending on the test automation server type, various actions are possible in the Overview of the automated test execution pop-up.

Execution Environment Statuses

Execution Environments Status

For more information about the execution environments status, visit the get channels page on the OpenTestFactory documentation.

A summary of the execution environments and their statuses is available in the Environments capsule of the Iteration and Suite sections of the test campaigns as soon as their execution plan contains automated tests that will be executed by a Squash Orchestrator server.

The possible statuses are represented by an icon and a tooltip:

Global Status	Icon and Tooltip	Examples of Possible Causes
Valid		All orchestrators linked to the test plan are reachable: - They have a valid token, either at the project level or at the automated execution server level. - The URLs provided in Squash TM are correct and allowed the orchestrators to be reached. - The orchestrators versions are compatible with the Squash TM version.
Error		All orchestrators linked to the test plan are unreachable: - The URLs provided in the Administration > Servers > Automated Execution Servers section do not allow the orchestrators to be reached or are incorrect. - The tokens provided (at the project or server level) are not valid. - The orchestrators are stopped. - The orchestrators have an outdated version.
Warning		The token for one or more orchestrators is not provided (neither at the project level nor at the automated server level).
		The test plan is linked to multiple projects, and at least one orchestrator linked to the test plan is unreachable.

If execution environments are found, you can click on the capsule to display a pop-up containing details about the execution environments (name, tags, status).

For the specific case of an automated test suite with tests coming from different projects and linked either to different orchestrators or to different namespaces of the orchestrator, the pop-up contains details of the execution environments by project:

Running automated tests with Squash Orchestrator

Token Authentication

The execution of automated tests from Squash TM uses temporary API tokens. Therefore, the property squash.rest-api.jwt.secret is required to run automated tests with Squash Orchestrator from Squash TM.

When the automated test is running with Squash Orchestrator, the Overview of the automated test execution pop-up displays the tests to be executed.

From this pop-up, it is also possible to view the environment tags which will be used to run the tests.
By default, the selected environment tags are those defined for the project. However, the user can set other values that will be used only for this execution.

The Available environments section enables the user to check that the selected combination matches at least one available execution environment. If not, the test cannot be run.

If environment variables are associated with the test automation server, they are displayed in this pop-up with the default value defined for the project. It is possible to modify their values only for this execution.

When an automated test suite is run with Squash Orchestrator, a progress bar shows the progress of test executions in the supervision pop-up. The UUID of the workflow(s) in charge of execution on the orchestrator are also displayed.

Focus

Closing the supervision pop-up is permanent. If automated executions are in progress, they will nevertheless continue, but you will not be able to view their real-time progress.

Execution order of tests

The only order assured by Squash is that, for a given Git repository in a given iteration/test suite, the tests will be executed in the order defined in Squash TM.
If a test iteration/suite contains automated tests belonging to multiple Git repositories, the order of execution of tests from one repository relative to the execution of tests from another repository is undefined.
If several test iterations/suites are launched from Squash TM (resp. the CI/CD pipeline), the order of execution of the tests of one iteration/suite relative to the execution of the tests of another iteration/suite is undefined. (Except in the trivial case where a suitable execution environment is available and the first suite/iteration was able to start before the Squash TM user – resp. the CI/CD pipeline – launches another one.)

Running automated tests with Squash TF

When the automated test is running with Squash TF, a progress bar shows the progress of the test executions in the Overview of the automated test execution pop-up.

Info

When executing with a Squash TF automated runtime server such as Jenkins, if it is configured to let you decide on the execution server, a pop-up allowing you to select the server opens before the execution starts.

Automated test suites

An automated suite is systematically created after the launch of automated tests. The suite is created at the iteration and test suite level. From this automated suite, it is possible to check:

• the overall execution status of the automated suite (depending on status of the associated executions);
• the details of the executions;
• the reports of all executions of the automated suite.