Sonarcloud not picking up line coverage from coverage.xml

[EdWade]

Template for a good new topic, formatted with Markdown:

• ALM used: GitHub

• CI system used: Travis CI

• Languages of the repository : PHP

• Steps to reproduce;
  We have setup a project using PHPUnit attempting to show code coverage in Sonarcloud. We have several projects in Sonarcloud all setup the same way, however only one has this issue;
  We run PHPUnit to generate a coverage.xml and a results.xml - checking the contents of the file we can see the lines of code we expect to be covered are, however when looking at Sonarcloud it does not show the lines as covered. Using identical config but passing the results in to a dockerised Sonarqube instance does show the lines as covered as expected. This uses the same sonar-project.properties file.

• Potential workaround;
  No workaround found as of yet.

[Nils_Werner]

Hi @EdWade,

Welcome to the community, can you share the CI pipeline and check the logs if there is any warning about processing the coverage file?

Best,
Nils

[EdWade]

Hey Nils - thank you!

The only log entry relating to code coverage is INFO: Importing /home/travis/build/{REDACTED}/{REDACTED}/codebase/tests/coverage.xml and other coverage is showing OK. We use Travis to run our CI Pipeline so are using the sonar-scanner command available when using the sonarcloud addon in our TravisCI

The codebase is a Drupal9 codebase so we have the following in our properties file;

sonar.sources=codebase/docroot/modules/custom
sonar.tests=codebase/docroot/modules/custom
sonar.exclusions=codebase/docroot/modules/custom/**/tests/**/*.php
sonar.test.inclusions=codebase/docroot/modules/custom/**/tests/**/*.php

Let me know if there’s anything else you need from me to help out!

[Nils_Werner]

Thanks for sharing, can you confirm, that the generated coverage file is not empty?

[EdWade]

Hey,
I can, yes - I have debugged the build on Travis and can confirm the file is generated and that when searching through the file manually, I can find the lines I expect to be covered showing as having coverage.

<line num="99" type="method" name="createListingGroups" visibility="protected" complexity="3" crap="3.01" count="3"/>
<line num="100" type="stmt" count="3"/>
<line num="102" type="stmt" count="3"/>
<line num="105" type="stmt" count="3"/>
<line num="108" type="stmt" count="3"/>
<line num="109" type="stmt" count="3"/>
<line num="111" type="stmt" count="0"/>
<line num="114" type="stmt" count="3"/>
<line num="115" type="stmt" count="3"/>
<line num="116" type="stmt" count="2"/>
<line num="119" type="stmt" count="2"/>
<line num="121" type="stmt" count="2"/>
<line num="124" type="stmt" count="3"/>

[Nils_Werner]

Hi @EdWade,

Can you double-check that the pipeline is passing the generated report to the scanner properly and that the scanner is configured correctly?

Also, could you share your pipeline and scanner properties with us? A full log would also help to clarify the issue.

Best,

[EdWade]

Hey,

Our Travis pipeline is as follows:

before_install:
    - phpenv config-rm xdebug.ini
    - composer self-update --2
    - mysqladmin -uroot create phpunit
    - pecl install pcov
install:
    - cd ./codebase
    - composer install --optimize-autoloader
script:
    - cp phpunit.xml.dist  phpunit.xml
    - ./vendor/bin/paratest --testsuite=our-testsuite-name --runner WrapperRunner --group ourgroupname --processes 5 --coverage-clover tests/coverage.xml --log-junit tests/results.xml
after_success:
    - cd ../
    - sonar-scanner

Our sonar-project.properties is as follows:

sonar.projectKey=our_project_key
sonar.organization=our_org

sonar.sources=codebase/docroot/modules/custom
sonar.tests=codebase/docroot/modules/custom
sonar.exclusions=codebase/docroot/modules/custom/**/tests/**/*.php
sonar.test.inclusions=codebase/docroot/modules/custom/**/tests/**/*.php

sonar.php.tests.reportPaths=codebase/tests/results.xml
sonar.php.coverage.reportPaths=codebase/tests/coverage.xml

sonar.sourceEncoding=UTF-8

I’m not seeing any errors in the logs relating to it picking up the coverage file and we see INFO: Importing /home/travis/build/redacted/redacted/codebase/tests/coverage.xml so it all looks to be picking up correctly as expected, we’re just not seeing the coverage picked up by Sonarcloud but is when using a Sonarqube docker instance.

Thanks for your support so far

[Nils_Werner]

Hi @EdWade,

sorry for the silence. We are currently considering how to investigate the problem further.

• Can you share a full log with us?
• Do you see the missing coverage only in feature branches or also on the main branch?

We are also working on improved logging to further simplify the investigation. To rule out possible problems here as well.

Best,

[EdWade]

Hey Nils,

Apologies for the delay.

Log of a Sonarcloud scan run sonarcloud.txt (12.2 KB)
We see the missing coverage on everything; feature branches and the main branch.

Let me know if there’s anything else you need to help with this.

Cheers,

Ed

[Csaba_Feher (Csaba Fehér)]

Hi @EdWade ,

Could you please give it a try and specify: sonar.php.tests.reportPath along with or instead of sonar.php.tests.reportPaths
I see this in the logs:

“No PHPUnit tests reports provided (see ‘sonar.php.tests.reportPath’ property”

I see you specified reportPaths which is also mentioned in the docs.

Test Execution Parameters | SonarCloud Docs

I would try it, and please let me know the results. ( if it is not working by default, I would also try to remove sonar.php.tests.reportPaths=codebase/tests/results.xml and just leave the values for sonar.php.tests.reportPath).

[EdWade]

YAY! It works!

Thank you Nils and Csaba for your assistance on this! A combination of incorrect key names, incorrectly setup code coverage filters. Appreciate the extra logging and comments on this thread.

Regards,
Ed