Parse the report

Now that your tests are running and generating reports, you need to convert those reports into a format that SquashTM can understand.

What you'll learn in this section:

Parse an xUnit XML test report from Robot Framework
Convert it to SquashTM's JSON format
Match test results to the automated test references you configured in SquashTM

Robot Framework and xUnit XML example

This section demonstrates parsing xUnit XML output from Robot Framework. If you're using a different test framework or report format (e.g., Cucumber JSON), you'll need to adapt the parser logic to match your specific report structure.

In the previous section, you configured a pipeline that runs Robot Framework and produces an xunit.xml report. In this section, you'll add a parser to the pipeline that converts xunit.xml into SquashTM's JSON format.

See the Complete Parser Script

You can view the complete parser script here: parse_robot_results.py.

Example: GitLab CI/CD + Robot Framework

Step 1: Create the parser script

Create a Python script that parses the xUnit XML file and generates SquashTM-compatible JSON.

Create a ci directory in your project and add the parser script:

your-project/
├── .gitlab-ci.yml
├── ci/
│   └── parse_robot_results.py    ← Create this file
├── tests/
│   └── login_validation_tests.robot
└── ...

File: ci/parse_robot_results.py

#!/usr/bin/env python3
"""
Parse xUnit/JUnit XML test results and generate a SquashTM-compatible JSON report.

This script extracts test results from xUnit/JUnit XML files and creates
a structured JSON report compatible with SquashTM API format.
"""

import base64
import json
import xml.etree.ElementTree as ET
from pathlib import Path


def encode_file_to_base64(file_path):
    """Encode a file to base64."""
    with open(file_path, 'rb') as f:
        return base64.b64encode(f.read()).decode('utf-8')


def parse_tests(xml_file):
    """Parse xUnit/JUnit XML file."""
    tree = ET.parse(xml_file)
    root = tree.getroot()

    tests = []
    suite_name = root.get('name', 'Suite')

    # Loop through all test cases
    for testcase in root.findall('.//testcase'):
        test_name = testcase.get('name', 'Test')
        classname = testcase.get('classname', suite_name)
        time_seconds = float(testcase.get('time', '0'))

        # Determine status
        failure = testcase.find('failure')
        error = testcase.find('error')
        skipped = testcase.find('skipped')

        if failure is not None:
            status = 'FAILURE'
            failure_message = failure.get('message', failure.text or 'Test failed')
            failure_details = [failure_message.strip()]
        elif error is not None:
            status = 'BLOCKED'
            failure_details = None
        elif skipped is not None:
            status = 'SKIPPED'
            failure_details = None
        else:
            status = 'SUCCESS'
            failure_details = None

        # Build reference as '<InnermostSuite>.<TestName>'
        suite_for_ref = classname.split('.')[-1]

        # Create test result
        result = {
            'reference': f"{suite_for_ref}.{test_name}",
            'status': status,
            'duration': round(time_seconds * 1000)  # Convert to milliseconds
        }

        # Add failure details if present
        if failure_details:
            result['failure_details'] = failure_details

        tests.append(result)

    return tests


def create_attachments():
    """Create attachments (test report files encoded in base64)."""
    attachments = []
    files = ['xunit.xml', 'report.html', 'log.html', 'output.xml']

    for file in files:
        if Path(file).exists():
            attachments.append({
                'name': file,
                'content': encode_file_to_base64(file)
            })

    return attachments


def main():
    """Main function."""
    tests = parse_tests('xunit.xml')

    attachments = create_attachments()

    report = {
        'automated_test_suite': {
            'attachments': attachments
        },
        'tests': tests
    }

    with open('result.json', 'w', encoding='utf-8') as f:
        json.dump(report, f, indent=2, ensure_ascii=False)

    print(f"{len(tests)} test(s) parsed")
    print(f"result.json created with {len(attachments)} attachment(s)")


if __name__ == '__main__':
    main()

How the parser works

The script performs several operations: it parses the xUnit XML file, creates attachments, and generates the SquashTM JSON.

Format-specific implementation

The following sections explain parsing logic specific to xUnit XML format. If you're using a different report format, these sections will help you understand what needs to be adapted for your format.

1. Parsing the xUnit XML structure

The parse_tests() function reads the xUnit format which contains <testcase> elements with attributes like name, classname, and time. Each test case may contain child elements: <failure>, <error>, or <skipped> to indicate the result.

2. Mapping test status

The parser maps xUnit results to SquashTM statuses:

<failure> element → FAILURE status in SquashTM
<error> element → BLOCKED status in SquashTM
<skipped> element → SKIPPED status in SquashTM
No failure/error/skipped → SUCCESS status in SquashTM

3. Building the automated test reference

The reference uniquely identifies each test in SquashTM:

Format: <SuiteName>.<TestName> (e.g., Login Validation Tests.Valid Login Passes)
This reference must match the automated test reference field configured in your SquashTM test cases

4. Creating attachments

Robot Framework-specific attachments

The following attachments are specific to Robot Framework. Adapt the file names and types to match your test framework's output files.

The create_attachments() function encodes report files in base64:

xunit.xml: Test report in xUnit format
report.html: HTML report generated by Robot Framework
log.html: Detailed test log
output.xml: Complete Robot Framework XML output

5. Extracting additional information

Duration: Converted from seconds (xUnit) to milliseconds (SquashTM)
Failure details: Extracted from the <failure> element's message or text content

6. Generating the JSON file

The script creates a result.json file containing the complete structure with tests and attachments, ready to be sent to SquashTM.

Example transformation

xUnit XML input (failed test):

<testcase classname="Login Validation Tests" name="Invalid Login Fails" time="0.456">
  <failure message="AssertionError: Expected login to fail but it succeeded">
    Login with invalid credentials should have been rejected.
  </failure>
</testcase>

SquashTM JSON output:

{
  "reference": "Login Validation Tests.Invalid Login Fails",
  "status": "FAILURE",
  "duration": 456,
  "failure_details": [
    "AssertionError: Expected login to fail but it succeeded"
  ]
}

For the complete JSON structure reference including optional fields like automated_test_suite and attachments, see the detailed format specification.

Sending to SquashTM

In the next section, you'll add the code to automatically send this result.json file to SquashTM via the API. For now, the file is simply generated and saved as an artifact.

Step 2: Add the parser job to the pipeline

Now add a new job to your .gitlab-ci.yml that runs the parser after the tests complete.

Open your .gitlab-ci.yml file and add a post stage and the parser job:

stages:
  - test
  - post  # Add this new stage

# ... (keep your existing robotframework-tests job)

parse-and-publish:
  stage: post
  needs:
    - job: robotframework-tests
      artifacts: true
  when: always
  allow_failure: false
  script:
    - python --version
    - python ci/parse_robot_results.py
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - result.json

What this does:

stage: post: Runs after the test stage completes
needs: Waits for the robotframework-tests job and downloads its artifacts (including xunit.xml). This ensures the parser job has access to the xunit.xml file generated by the test job.
when: always: Runs even if tests failed (we want to parse all results), ensuring all results are captured.
allow_failure: false: Fails the pipeline if parsing fails
script: Executes the Python parser script
artifacts: Saves the generated result.json file for inspection and traceability

Complete pipeline file

At this stage, your complete .gitlab-ci.yml should look like this:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "web"'
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
    - when: never

variables:
  ROBOT_VERSION: "7.4.1"

default:
  image: python:3.14-slim

stages:
  - test
  - post

robotframework-tests:
  stage: test
  allow_failure: true
  script:
    - python --version
    - pip install --upgrade pip --root-user-action=ignore
    - pip install robotframework==${ROBOT_VERSION} --root-user-action=ignore
    - robot --xunit xunit.xml ./tests/
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - output.xml
      - log.html
      - report.html
      - xunit.xml
    reports:
      junit: xunit.xml

parse-and-publish:
  stage: post
  needs:
    - job: robotframework-tests
      artifacts: true
  when: always
  allow_failure: false
  script:
    - python --version
    - python ci/parse_robot_results.py
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - result.json

Step 3: Commit and verify

Commit your changes and push to GitLab:

git add ci/parse_robot_results.py .gitlab-ci.yml
git commit -m "Add test results parser"
git push

Verify the parser

Go to your GitLab project
Navigate to Build → Pipelines
Click on the latest pipeline
You should now see two jobs: robotframework-tests and parse-and-publish
Click on the parse-and-publish job to see the parser output
Once complete, download the result.json artifact to inspect the parsed results

What you've accomplished

Your CI/CD pipeline now:

Runs automated tests
Parses test results into SquashTM's JSON format
Generates a result.json file with all test results
Matches test results to automated test references

Next step

Now that you have parsed test results in SquashTM's JSON format, the final step is to publish these results to your SquashTM instance via the API.

In the next section, you'll learn how to add a job that sends the result.json file to SquashTM.

Proceed to Publish to SquashTM