Provide test results from multiple steps

Which versions are you using?

  • SonarQube 8.2
  • Azure DevOps Server 2019

What are you trying to achieve?
We have a YAML build pipeline with two jobs. Is there any chance to get the test results from both steps reported to SonarQube in one analyses?

In 1st job the

  • Prepare SonarQube analyses
  • build and unit tests
  • publish (unit) test results
  • Run SonarAnalyses and Post Quality Gate Result
  • publish build artifacts

In the 2nd job (other system, other agent pool)

  • download build artifacts
  • run integration tests
  • publish (integration) test results

In SonarQube (obviously) only the unit test results and code coverage are available.
We would like to have in SonarQube test results and code coverage of all tests from both jobs.

1 Like

Hi Volker,

There is currently no way to add additional coverage reports after the scan complete.

SonarQube will accept multiple test coverage reports (assuming they are in supported formats), but they need to both be available at the time the scan is run to be imported.

If altering your pipeline is feasible, you could run your SonarQube scan after all of the coverage reports are produced. By doing this, you will not get the Quality Gate status until after you run your integration test.

Brian

Thanks @bcipollone. Yes, the integration test results are available as trx and are shown on the test results of Azure DevOps Server - but not in SonarQube.

Let me repeat with my own words:

  • Prepare Analyses and Run Code Analysis have to be in the same job surrounding the build.
  • I would have to do everything in one job as each job is running on an arbitrary agent which fulfill its requirements.
  • That said if I have different system requirements I would need to copy the tests to the specialised integration test hardware, start them during the only job and copy the test results back and publish. This would block the build agent during integration tests. Further it would limit setting up a pool of multiple integration test systems.
  • Otherwise I could do it with 3 jobs: 1) build, 2) run integration tests, 3) run SonarQube-Steps with build, unit tests and publish integration test results. This would require to build twice which increases the total runtime of pipeline .

Hi Volker,

Running in three steps would make sense but you do not need to run the tests again, just provide the results to the scanner from the previous runs.

Understood this is may not be optimal. Weighing overall time vs. coverage for integration test results is something you’ll need to consider.

Good luck.

Brian