Skip to content

CI/CD Integration of Automated Tests

Integrate automated test results from your CI/CD pipeline into SquashTM to track execution trends, analyze failures, and maintain a testing history.

Why integrate CI/CD pipeline results into SquashTM?

By publishing automated test results to SquashTM, you gain:

  • Centralized test results: View all test executions (manual and automated) in one place
  • Execution history: Track test trends over time and identify flaky tests
  • Failure analysis: Quickly identify which tests failed, with detailed error messages and attachments
  • Issue tracking: Link failed tests to bugs in your bug tracker (Jira, GitLab, etc.)
  • Traceability: Link automated test results to requirements and test cases
  • Reporting: Generate reports combining manual and automated test results

Once your CI/CD integration is complete, you'll be able to view and analyze automated test results in SquashTM.
See Analyze CI/CD Results for details on how to explore your automated test executions in SquashTM.

What you'll learn

  • Set up SquashTM to receive automated test results
  • Configure your CI/CD pipeline to run tests and generate reports
  • Parse test results into SquashTM's JSON format
  • Publish results and test reports to SquashTM via API

Additionally, you'll find a reference to the page that explains how to analyze CI/CD results in SquashTM.

Who is this for?

This guide covers multiple scenarios:

  • You already have automated tests and a CI/CD pipeline - Integrate existing tests with SquashTM
  • You have automated tests but no CI/CD pipeline yet - Set up a pipeline and integrate with SquashTM
  • You're starting from scratch - Build a complete automated testing workflow

GitLab CI and Robot Framework example

This guide uses GitLab CI as the CI/CD platform and Robot Framework with xUnit XML reports as examples. The steps and concepts apply to any CI/CD platform (Jenkins, GitHub Actions, Azure DevOps, etc.) and test framework, but you'll need to adapt:

  • CI/CD syntax: Pipeline configuration syntax varies by platform (e.g., .gitlab-ci.yml vs Jenkinsfile vs GitHub Actions workflows)
  • Parser logic: Report parsing must match your test framework's output format (e.g., Cucumber JSON)

What you need

  • SquashTM instance with administrator access
  • CI/CD platform (i.e. GitLab CI) with an execution environment for your tests
  • Basic knowledge of CI/CD concepts, YAML/JSON, and command line
  • Automated tests in any framework that generates parsable output (we use Robot Framework with xUnit XML in this guide)

Cloud Premium limitation

On SquashTM Cloud Premium, the results import endpoint used by this guide is not available. Use SquashTM Cloud Ultimate or SquashTM Server to publish via the API.

Don't have automated tests yet?

Use our Robot Framework test suite as a starting point: login_validation_tests.robot.

Time to complete

Following this guide step-by-step (including reading, configuration, commits, and verification):

  • Data preparation in SquashTM: 15-20 minutes
  • Pipeline configuration: 15-20 minutes
  • Parser setup: 10-15 minutes
  • Publishing configuration: 10-15 minutes

Total time: Approximately 1 hour

Guide structure

This guide is organized into sequential steps:

  1. Prepare SquashTM - Configure SquashTM to receive test results
  2. Run automated tests in CI/CD - Configure your pipeline to run tests
  3. Parse the report - Convert test outputs to SquashTM format
  4. Publish to SquashTM - Send results to SquashTM via API
  5. Troubleshooting - Common issues and solutions

Each step builds on the previous one, so it is recommended to follow them in order.

Example: GitLab CI + Robot Framework

Throughout this guide, we use a complete working example with:

  • CI/CD platform: GitLab CI/CD
  • Test framework: Robot Framework
  • Programming language: Python (for the parser)

However, the concepts and approaches apply to any CI/CD platform and test framework. You can adapt the examples to your specific tools.

Architecture overview

GitLab Repository & CI/CD
Automated Tests
(Robot Framework)
↓ Execute
Pipeline Job
Run Tests

Artifacts
• xunit.xml
• report.html
• log.html
↓ Parse
Parser Job
Convert & Publish
POST JSON + Attachments
———>
SquashTM
REST API
/import/results/{iteration_id}
↓ Create
Automated Test Suite
↓ Contains
Automated Executions
• Status (Success/Failure)
• Execution date
• Test details
↓ With
Attachments
HTML reports, logs, screenshots

How it works:

  1. Run tests: Your CI/CD pipeline executes automated tests and generates a test report (xUnit/JUnit XML)
  2. Parse results and prepare data: A parser script converts the XML report to SquashTM's JSON format and collects attachments (logs, screenshots, etc.)
  3. Publish to SquashTM: Results are sent to SquashTM via REST API
  4. View in SquashTM: An automated test suite is created in your iteration, with all test results and attachments

Learn more: Analyze CI/CD Results

Complete Working Example

You can view the complete working example repository here: gitlab-automated-test-integration.
The corresponding test plan is available here, after logging in as a guest with the username Squash (the password is indicated on the login page).

Next steps

Ready to get started? Begin with Prepare SquashTM to configure your SquashTM environment.