We have CI running in drone from GitHub. There are multiple drone pipelines running PHP unit tests. Each pipeline runs unit tests with some environment combination - e.g. different backend database, different storage like local filesystem, samba… I am trying to get unit test code coverage working with SonarCloud.
See PR https://github.com/owncloud/core/pull/37931
Example drone log output https://drone.owncloud.com/owncloud/core/26880/35/10
As each pipeline finishes, it uses docker image sonarsource/sonar-scanner-cli to do its thing and upload the clover XML coverage information.
On GitHub lots of posts keep coming for each individual unit test result. In https://sonarcloud.io/dashboard?id=owncloud_core&pullRequest=37931 it does report a coverage result. But that seems to be just the individual coverage from the latest unit test pipeline that finished.
I would like a total coverage report that shows the total lines covered (by any unit test run). e.g. an individual unit test run might cover 50%, but a slightly different 50% to another test run. Combined, they might provide 60% coverage.
How can I get SonarCloud to “wait” until it has received all the clover XML coverage reports, and then do a single analysis and report the combined coverage?
Hi @phil-davis and welcome to the community !
I’m afraid this is not currently supported ‘out-of-the-box’.
Have you tried maybe to merge those coverage files together, and then do a unique analysis ? Not sure if it can work, but worth the try maybe.
The challenge is that each drone pipeline runs in its own drone agent “process”. There could be multiple on the same real system, but actually once there are more than 4 or 8 of them then they will definitely be on different real systems. So there is no hope to “just dump the coverage reports somewhere in the filesystem” and then upload them in one go at the end.
I will have to upload each of them to a common online storage, then in a “last pipeline” upload them all together.
If there are multiple coverage reports and I upload them all in the same sonarsource/sonar-scanner-cli run, will SonarCloud calculate them “all together”?
At least the
sonar.php.coverage.reportPaths property allows you to pass a list of comma separated value of coverage paths, you can give it a try.
I am going to try! I will post back here a link to whatever solution I manage to do.
It would be even nicer if
sonar.php.coverage.reportPaths accepted wildcards, so I could put something like
reports/clover*.xml - but I think I am going to have to have a script gets the expanded list of
reports/clover*.xml file names from the
reports directory and puts the expanded list into
sonarsource/sonar-scanner-cli does its thing.
I am getting confused about the names of parameters in sonar-project.properties
https://docs.sonarqube.org/latest/analysis/analysis-parameters/ talks about
But various posts with questions talk about
sonar.inclusions - which is not mentioned on that docs page?
Which is the correct one to define in order to select the code for analysis?
That doc talks about the online UI form that you can fill in for the project. It does not mention lots of parameters that can be set in sonar-project.properties
https://docs.sonarqube.org/latest/analysis/analysis-parameters/ has the parameter names to put in sonar-project.properties but it does not mention the word
inclusions at all. And the only parameter containing the word “exclusions” is
So I am guessing a bit from internet searches that
sonar.exclusions do something.
sonar.sources needs to be filled with the main path to your sources.
Then, after having set that, you can setup a small subset of files with sonar.inclusions, or exclude a subset with sonar.exclusions, so yes, they definitely do something.
What do you want to achieve with those so far ? Is that part of you initial question ?
I am working on this in PR https://github.com/owncloud/core/pull/37946/files
Here https://drone.owncloud.com/owncloud/core/26996/51/7 I fetch the
owncloud/core software and get all the unit test results into.
Then the sonarcloud docker image runs https://drone.owncloud.com/owncloud/core/26996/51/8
I do get a result, https://sonarcloud.io/dashboard?id=owncloud_core&pullRequest=37946 - but in the output of the drone sonarcloud run I see stuff like:
WARN: Failed to resolve 1 include/require statements like '../../../../3rdparty/autoload.php' from 'bin'
INFO: 1485/1485 source files have been analyzed
INFO: No PHPUnit test report provided (see 'sonar.php.tests.reportPath' property)
INFO: Analyzing PHPUnit coverage report: results/clover-phpunit-php7.2-mariadb10.2.xml
WARN: Could not resolve 5083 file paths in clover-phpunit-php7.2-mariadb10.2.xml, first unresolved path: /drone/src/config/config.php
But, for example,
/drone/src/config/config.php is there.
I tried defining
SONAR_SCANNER_OPTS to be
-Xdebug - I was hoping that would reveal the 5083 file paths that could not be resolved, and I would then get some clue about that. But there is not any more output than normal.
I was hoping to get a code coverage of about 65%, like previously calculated - but I am getting 50.7%. So I am hoping that the “5083 unresolved” will be part of the problem.
Any clues from anyone?
We ended up doing code in drone to sync the coverage report from each unit test pipeline to an S3 cache storage, then sync all the coverage reports back from the cache in a last drone pipeline. That last pipeline puts the ccode of the PR branch in place, so that the sonar analysis can see the code, and runs the
sonarsource/sonar-scanner-cli docker image, which “does its thing” and puts a report for the PR online.
See https://github.com/owncloud/core/pull/37984 https://github.com/owncloud/core/pull/37982
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.