We have CI running in drone from GitHub. There are multiple drone pipelines running PHP unit tests. Each pipeline runs unit tests with some environment combination - e.g. different backend database, different storage like local filesystem, samba… I am trying to get unit test code coverage working with SonarCloud.
As each pipeline finishes, it uses docker image sonarsource/sonar-scanner-cli to do its thing and upload the clover XML coverage information.
On GitHub lots of posts keep coming for each individual unit test result. In https://sonarcloud.io/dashboard?id=owncloud_core&pullRequest=37931 it does report a coverage result. But that seems to be just the individual coverage from the latest unit test pipeline that finished.
I would like a total coverage report that shows the total lines covered (by any unit test run). e.g. an individual unit test run might cover 50%, but a slightly different 50% to another test run. Combined, they might provide 60% coverage.
How can I get SonarCloud to “wait” until it has received all the clover XML coverage reports, and then do a single analysis and report the combined coverage?
The challenge is that each drone pipeline runs in its own drone agent “process”. There could be multiple on the same real system, but actually once there are more than 4 or 8 of them then they will definitely be on different real systems. So there is no hope to “just dump the coverage reports somewhere in the filesystem” and then upload them in one go at the end.
I will have to upload each of them to a common online storage, then in a “last pipeline” upload them all together.
If there are multiple coverage reports and I upload them all in the same sonarsource/sonar-scanner-cli run, will SonarCloud calculate them “all together”?
I am going to try! I will post back here a link to whatever solution I manage to do.
It would be even nicer if sonar.php.coverage.reportPaths accepted wildcards, so I could put something like reports/clover*.xml - but I think I am going to have to have a script gets the expanded list of reports/clover*.xml file names from the reports directory and puts the expanded list into sonar.php.coverage.reportPaths before sonarsource/sonar-scanner-cli does its thing.
That doc talks about the online UI form that you can fill in for the project. It does not mention lots of parameters that can be set in sonar-project.properties
https://docs.sonarqube.org/latest/analysis/analysis-parameters/ has the parameter names to put in sonar-project.properties but it does not mention the word inclusions at all. And the only parameter containing the word “exclusions” is sonar.scm.exclusions.disabled
So I am guessing a bit from internet searches that sonar.inclusions and sonar.exclusions do something.
sonar.sources needs to be filled with the main path to your sources.
Then, after having set that, you can setup a small subset of files with sonar.inclusions, or exclude a subset with sonar.exclusions, so yes, they definitely do something.
What do you want to achieve with those so far ? Is that part of you initial question ?
WARN: Failed to resolve 1 include/require statements like '../../../../3rdparty/autoload.php' from 'bin'
INFO: 1485/1485 source files have been analyzed
INFO: No PHPUnit test report provided (see 'sonar.php.tests.reportPath' property)
INFO: Analyzing PHPUnit coverage report: results/clover-phpunit-php7.2-mariadb10.2.xml
WARN: Could not resolve 5083 file paths in clover-phpunit-php7.2-mariadb10.2.xml, first unresolved path: /drone/src/config/config.php
But, for example, /drone/src/config/config.php is there.
I tried defining SONAR_SCANNER_OPTS to be -Xdebug - I was hoping that would reveal the 5083 file paths that could not be resolved, and I would then get some clue about that. But there is not any more output than normal.
I was hoping to get a code coverage of about 65%, like previously calculated - but I am getting 50.7%. So I am hoping that the “5083 unresolved” will be part of the problem.
We ended up doing code in drone to sync the coverage report from each unit test pipeline to an S3 cache storage, then sync all the coverage reports back from the cache in a last drone pipeline. That last pipeline puts the ccode of the PR branch in place, so that the sonar analysis can see the code, and runs the sonarsource/sonar-scanner-cli docker image, which “does its thing” and puts a report for the PR online.