No Python code coverage even though report is being processed

(Philip Colmer) #1

I have an open source project on GitHub and I’ve configured SonarCloud to analyse it. I’m running the tests on my computer to generate the XML coverage report and I’m then running sonar-scanner.

The scanner appears to be happy to import the report:

<snip>
INFO: Python test coverage
INFO: Parsing report '/home/philip/Documents/GitHub/sd-webhook-framework/coverage.xml'
INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=587ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=226ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=26ms
<snip>

but SonarCloud is showing 0.0% coverage (even though the tests show 100% coverage), suggesting that the coverage report is not being processed correctly.

What have I misunderstood/missed out?

Thanks.

Sonar Python not showing test coverage
(Fabrice Bellingard) #2

Hi Philip,

have you solved your problem? If not, can you try running the analysis in debug mode (i.e. passing sonar.verbose=true parameter).

(Philip Colmer) #3

Hi @Fabrice_Bellingard

No, I haven’t solved the problem. If I enable debug mode, I was getting messages that there was an invalid directory path. I then re-ran the Python tests to generate a new coverage.xml file. That has eliminated the path messages and just gives me this:

10:41:38.976 DEBUG: Sensors : JavaXmlSensor -> HTML -> XML Sensor -> Python Squid Sensor -> Cobertura Sensor for Python coverage -> PythonXUnitSensor -> JaCoCo XML Report Importer -> JavaSecuritySensor -> CSharpSecuritySensor -> PhpSecuritySensor
10:41:38.978 INFO: Sensor JavaXmlSensor [java]
10:41:38.983 INFO: 1 source files to be analyzed
10:41:39.023 INFO: Load project repositories
10:41:39.053 DEBUG: GET 200 https://sonarcloud.io/batch/project.protobuf?key=linaro-its_sd-webhook-framework | time=30ms
10:41:39.073 INFO: Load project repositories (done) | time=51ms
10:41:39.073 DEBUG: 'coverage.xml' generated metadata with charset 'UTF-8'
10:41:39.591 INFO: Sensor JavaXmlSensor [java] (done) | time=614ms
10:41:39.591 INFO: 1/1 source files have been analyzed
10:41:39.592 INFO: Sensor HTML [web]
10:41:39.626 INFO: Sensor HTML [web] (done) | time=34ms
10:41:39.627 INFO: Sensor XML Sensor [xml]
10:41:39.631 INFO: 1 source files to be analyzed
10:41:39.813 DEBUG: Count lines in file:///home/philip/Documents/GitHub/sd-webhook-framework/coverage.xml
10:41:39.933 INFO: Sensor XML Sensor [xml] (done) | time=307ms
10:41:39.934 INFO: 1/1 source files have been analyzed
10:41:39.934 INFO: Sensor Python Squid Sensor [python]
10:41:39.942 DEBUG: 'app.py' generated metadata with charset 'UTF-8'
10:41:40.195 DEBUG: 'rt_handlers/rt206.py' generated metadata with charset 'UTF-8'
10:41:40.277 DEBUG: 'sd_webhook_automation/config.py' generated metadata with charset 'UTF-8'
10:41:40.280 DEBUG: Not enough content in 'sd_webhook_automation/config.py' to have CPD blocks, it will not be part of the duplication detection
10:41:40.355 DEBUG: 'sd_webhook_automation/custom_fields.py' generated metadata with charset 'UTF-8'
10:41:40.475 DEBUG: 'sd_webhook_automation/shared_sd.py' generated metadata with charset 'UTF-8'
10:41:40.627 DEBUG: 'tests/test_app.py' generated metadata with charset 'UTF-8'
10:41:40.734 DEBUG: 'tests/test_config.py' generated metadata with charset 'UTF-8'
10:41:40.812 DEBUG: 'tests/test_custom_fields.py' generated metadata with charset 'UTF-8'
10:41:40.915 DEBUG: 'tests/test_rt206.py' generated metadata with charset 'UTF-8'
10:41:40.991 DEBUG: 'tests/test_shared_sd.py' generated metadata with charset 'UTF-8'
10:41:41.137 INFO: Sensor Python Squid Sensor [python] (done) | time=1203ms
10:41:41.137 INFO: Sensor Cobertura Sensor for Python coverage [python]
10:41:41.140 DEBUG: Using pattern 'coverage-reports/*coverage-*.xml' to find reports
10:41:41.239 WARN: No report was found for sonar.python.coverage.reportPaths using pattern coverage-reports/*coverage-*.xml
10:41:41.239 DEBUG: Using pattern 'coverage.xml' to find reports
10:41:41.312 INFO: Python test coverage
10:41:41.316 INFO: Parsing report '/home/philip/Documents/GitHub/sd-webhook-framework/coverage.xml'
10:41:41.344 INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=207ms

The analysis results are definitely getting updated as I’ve been fixing code smells, re-running the analysis and the web page has been updating. The only thing I can’t get to work is the code coverage.

Thanks.

(Fabrice Bellingard) #4

Hi Philip,

from what I see, the problem might come from the fact that the coverage report:

  • is generated on one machine, from a repository located in the following path: c/Users/phili/Documents/GitHub/
    • I can see this because the coverage report is imported in SonarCloud (coverage.xml file) so I could see that this path is used inside this report file
    • Note for the future: those files should be excluded - there is no point in analyzing them
  • but is consumed on another one: /home/philip/Documents/GitHub/sd-webhook-framework/coverage.xml
    • This is why we see in your logs

Because of this, the paths do not match and SonarCloud probably does not manage to connect the dots.

I’d suggest that you run your tests with coverage and then trigger the analysis just afterwards, from the same location. Hopefully this is what will solve your issue.

(Philip Colmer) #5

Thanks, @Fabrice_Bellingard. It seems to be a side-effect of using Windows Subsystem for Linux. py.test digs beneath the symlink I’m using to get the “real” path hence the disparity.

If I switch to the explicit path then SonarCloud seems to be a bit happier in that coverage figures are now getting uploaded, although I’ve now got to figure out how to get SonarCloud to only look at the actual source code and not the test files as that is resulting in a lower coverage than I’ve actually got. I must still be missing something obvious :slight_smile:

(Philip Colmer) #6

Solved the challenge of getting SonarCloud to look at just the source code.

Just in case anyone else needs to know, sonar.tests is a comma-separated list of test folders and sonar-sources is a comma-separated list of source folders. In my use-case, I have a single source file at the root (app.py) but SonarCloud is happy to have that included in the list of source “folders”, e.g.:

-Dsonar.sources=app.py,rt_handlers,sd_webhook_automation -Dsonar.tests=tests

and I now have 100% code coverage being reported on SonarCloud.

(Fabrice Bellingard) #7

Yeah! :metal:

Yes indeed, sources and tests must be in separate distinct folders, so that test files are not considered as “to be covered by tests” :wink: