Import Automated Test Results from Pipelines

SquashTM allows you to import the results of automated tests executed in a CI/CD pipeline.

This is done via an API call (see details), which will create a new automated test suite linked to the targeted iteration.

Prerequisites

In order to import test results, an iteration must already exist in SquashTM and its execution plan must be populated with the tests for which results will be imported.

Each of these tests must have a unique (in the iteration's execution plan) automated test reference.

All of these steps can be done via the API if needed:

• Create test cases: POST request to /test-cases with the automated_test_reference field filled;
• Create a campaign: POST request to /campaigns;
• Create an iteration: POST request to /campaigns/{campaign_id}/iterations;
• Add tests to an iteration: POST request to /iterations/{iteration_id}/test-plan.

For more details on these requests, refer to the API documentation available in SquashTM.

Mapping Information

Results will be reintegrated into SquashTM by matching the automated test reference provided in the API request with that of the test in the execution plan of the selected iteration.
If some test cases use datasets, mapping will rely on both the automated test reference and the dataset name associated with the ITPI.

The following information can be reported:

• the test status;
• the execution duration of the test;
• the messages of failed assertions;
• the attachments related to test executions;
• the custom fields related to test executions.

It is also possible to add attachments at the test suite level.
The test suite status will be automatically calculated based on the individual test results, but can also be overridden if needed.

API Endpoint Documentation

POST /import/results/{iteration_id}

🔸 Path Parameter

🔸 Request Body

Note: There are two types of required fields:
- 🔴: Fields that block the import if they are missing.
- 🟡: Fields that will result in their current object being rejected if they are invalid or missing, but not blocking the rest of the import.

Content-Type: application/json
Authentication: Bearer token

Example:

{
  "automated_test_suite": {
    "attachments": [
      {
        "name": "report.html",
        "content": "Base64-encoded content here..."
      }
    ]
  },
  "tests": [
    {
      "reference": "testSuite.testClass.testRef1",
      "dataset_name": "admin_dataset",
      "status": "FAILURE",
      "duration": 35000,
      "failure_details": ["1 is not equal to 2"],
      "attachments": [
        {
          "name": "screenshot.png",
          "content": "Base64-encoded content here..."
        }
      ],
      "test_steps": [
        {
            "status": "SUCCESS"
        },
        {
          "status": "FAILURE",
            "attachments": [
              {
                "name": "step_screenshot.png",
                "content": "Base64-encoded content here..."
              }
            ]
        }
      ],
      "custom_fields" : [ {
        "code" : "NUMERIC",
        "value" : 200
      }, {
        "code" : "TEXT",
        "value" : "description"
      }, {
        "code" : "CHECKBOX",
        "value" : "true"
      }, {
        "code" : "TAGS_RELATED",
        "value" : ["tag1", "tag2"]
      } ]
    }
  ]
}

🔸 Responses

{
  "iteration_id": 1234,
  "tests": [
    {
      "test_case_id": null,
      "reference": "testSuite.testClass.testRef1",
      "error": "No test found with this reference."
    }
  ]
}

{
  "fieldValidationErrors": [
    {
      "objectName": "post-execution-result-import",
      "fieldName": "tests[0].reference",
      "fieldValue": null,
      "errorMessage": "The reference attribute is required"
    }
  ] 
}