How to get total coverage from multiple unit test runs

We have CI running in drone from GitHub. There are multiple drone pipelines running PHP unit tests. Each pipeline runs unit tests with some environment combination - e.g. different backend database, different storage like local filesystem, samba… I am trying to get unit test code coverage working with SonarCloud.

See PR

Example drone log output

As each pipeline finishes, it uses docker image sonarsource/sonar-scanner-cli to do its thing and upload the clover XML coverage information.

On GitHub lots of posts keep coming for each individual unit test result. In it does report a coverage result. But that seems to be just the individual coverage from the latest unit test pipeline that finished.

I would like a total coverage report that shows the total lines covered (by any unit test run). e.g. an individual unit test run might cover 50%, but a slightly different 50% to another test run. Combined, they might provide 60% coverage.

How can I get SonarCloud to “wait” until it has received all the clover XML coverage reports, and then do a single analysis and report the combined coverage?

1 Like

Hi @phil-davis and welcome to the community !

I’m afraid this is not currently supported ‘out-of-the-box’.

Have you tried maybe to merge those coverage files together, and then do a unique analysis ? Not sure if it can work, but worth the try maybe.


The challenge is that each drone pipeline runs in its own drone agent “process”. There could be multiple on the same real system, but actually once there are more than 4 or 8 of them then they will definitely be on different real systems. So there is no hope to “just dump the coverage reports somewhere in the filesystem” and then upload them in one go at the end.

I will have to upload each of them to a common online storage, then in a “last pipeline” upload them all together.

If there are multiple coverage reports and I upload them all in the same sonarsource/sonar-scanner-cli run, will SonarCloud calculate them “all together”?

Hi @phil-davis,

At least the sonar.php.coverage.reportPaths property allows you to pass a list of comma separated value of coverage paths, you can give it a try.


I am going to try! I will post back here a link to whatever solution I manage to do.
It would be even nicer if sonar.php.coverage.reportPaths accepted wildcards, so I could put something like reports/clover*.xml - but I think I am going to have to have a script gets the expanded list of reports/clover*.xml file names from the reports directory and puts the expanded list into sonar.php.coverage.reportPaths before sonarsource/sonar-scanner-cli does its thing.

1 Like

I am getting confused about the names of parameters in talks about sonar.sources

But various posts with questions talk about sonar.inclusions - which is not mentioned on that docs page?

Which is the correct one to define in order to select the code for analysis?

You can find more information about inclusions / exclusions in that page :

That doc talks about the online UI form that you can fill in for the project. It does not mention lots of parameters that can be set in has the parameter names to put in but it does not mention the word inclusions at all. And the only parameter containing the word “exclusions” is sonar.scm.exclusions.disabled

So I am guessing a bit from internet searches that sonar.inclusions and sonar.exclusions do something.

sonar.sources needs to be filled with the main path to your sources.

Then, after having set that, you can setup a small subset of files with sonar.inclusions, or exclude a subset with sonar.exclusions, so yes, they definitely do something.

What do you want to achieve with those so far ? Is that part of you initial question ?

Thanks !

I am working on this in PR
Here I fetch the owncloud/core software and get all the unit test results into.

Then the sonarcloud docker image runs

I do get a result, - but in the output of the drone sonarcloud run I see stuff like:

WARN: Failed to resolve 1 include/require statements like '../../../../3rdparty/autoload.php' from 'bin'
INFO: 1485/1485 source files have been analyzed
INFO: No PHPUnit test report provided (see 'sonar.php.tests.reportPath' property)
INFO: Analyzing PHPUnit coverage report: results/clover-phpunit-php7.2-mariadb10.2.xml
WARN: Could not resolve 5083 file paths in clover-phpunit-php7.2-mariadb10.2.xml, first unresolved path: /drone/src/config/config.php

But, for example, /drone/src/config/config.php is there.

I tried defining SONAR_SCANNER_OPTS to be -Xdebug - I was hoping that would reveal the 5083 file paths that could not be resolved, and I would then get some clue about that. But there is not any more output than normal.

I was hoping to get a code coverage of about 65%, like previously calculated - but I am getting 50.7%. So I am hoping that the “5083 unresolved” will be part of the problem.

Any clues from anyone?

We ended up doing code in drone to sync the coverage report from each unit test pipeline to an S3 cache storage, then sync all the coverage reports back from the cache in a last drone pipeline. That last pipeline puts the ccode of the PR branch in place, so that the sonar analysis can see the code, and runs the sonarsource/sonar-scanner-cli docker image, which “does its thing” and puts a report for the PR online.


This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.