Sonarqube python code coverage show 0%

arajan · August 19, 2020, 5:15pm

My firm is running SonarQube via SonarQube Scanner for Jenkins. I am having an issue where SonarQube shows 0% code coverage for my Python project.

SonarQube: 7.9.3
SonarQube Scanner for Jenkins: 2.11
Jenkins: 2.222.4
Python: 3.7.4

I have used both nose2 and pytest to run the tests (settled on pytest). The tests are running to completion and the coverage.xml shows ~68% coverage. However, SonarQube still shows 0% code coverage.

Here are the SonarQube Scanner Analysis Properties

sonar.projectKey=project:cpv_python
sonar.projectName=cpv_python
sonar.projectVersion=1.0
sonar.sources=services
sonar.language=py
sonar.sourceEncoding=UTF-8
# Test Results
sonar.python.xunit.reportPath=services/tests/nosetests.xml
# Coverage
sonar.python.coverage.reportPaths=services/tests/coverage.xml

The code being executed

echo "==============Executing unit tests============="
cd services/
pytest -v -o junit_family=xunit1 --cov=. --cov-report xml:tests/coverage.xml  --junitxml=tests/nosetests.xml
#nosetests -sv --with-xunit --xunit-file=nosetests.xml --with-xcoverage --xcoverage-file=coverage.xml
echo "==============Finished unit tests execution. Starting integration tests==============="
behave -f=json.pretty -o Cucumber.json
python -m behave2cucumber -i Cucumber.json -o Cucumber.json
echo "==============Finished integration tests execution============="

Result of the executed code:

============================= test session starts ==============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 -- /root/anaconda3/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/cpv/workspace/cpv-python-local-build/services/.hypothesis/examples')
rootdir: /home/cpv/workspace/cpv-python-local-build/services
plugins: hypothesis-5.5.4, astropy-header-0.1.2, openfiles-0.4.0, remotedata-0.3.2, arraydiff-0.3, doctestplus-0.5.0, cov-2.10.1
collecting ... collected 51 items

tests/test_LJ_plot_api.py::APITests::test_LJ_plot_XY_notequallength PASSED [  1%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_X_notdatetime PASSED   [  3%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_Layout PASSED       [  5%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_Nelson PASSED       [  7%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_bgcolor PASSED      [  9%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_title PASSED        [ 11%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_xlabel PASSED       [ 13%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_xlim PASSED         [ 15%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_ylabel PASSED       [ 17%]
tests/test_LJ_plot_api.py::APITests::test_LJ_plot_no_ylim PASSED         [ 19%]
tests/test_box_plot_api.py::APITests::test_box_plot_no_Layout PASSED     [ 21%]
tests/test_box_plot_api.py::APITests::test_box_plot_no_bgcolor PASSED    [ 23%]
tests/test_box_plot_api.py::APITests::test_box_plot_no_num_boxes PASSED  [ 25%]
tests/test_box_plot_api.py::APITests::test_box_plot_no_title PASSED      [ 27%]
tests/test_box_plot_api.py::APITests::test_box_plot_no_ylabel PASSED     [ 29%]
tests/test_box_plot_api.py::APITests::test_box_plot_no_ylim PASSED       [ 31%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_XY_notequallength PASSED [ 33%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_X_notdatetime PASSED [ 35%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_Layout PASSED     [ 37%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_bgcolor PASSED    [ 39%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_title PASSED      [ 41%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_xlabel PASSED     [ 43%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_xlim PASSED       [ 45%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_ylabel PASSED     [ 47%]
tests/test_imr_plot_api.py::APITests::test_IMR_plot_no_ylim PASSED       [ 49%]
tests/test_kde_plot_api.py::APITests::test_KDE_plot_no_Layout PASSED     [ 50%]
tests/test_kde_plot_api.py::APITests::test_KDE_plot_no_bgcolor PASSED    [ 52%]
tests/test_kde_plot_api.py::APITests::test_KDE_plot_no_title PASSED      [ 54%]
tests/test_kde_plot_api.py::APITests::test_KDE_plot_no_xlabel PASSED     [ 56%]
tests/test_kde_plot_api.py::APITests::test_KDE_plot_no_xlim PASSED       [ 58%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_XY_notequallength PASSED [ 60%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_X_notdatetime PASSED [ 62%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_Layout PASSED [ 64%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_Nelson PASSED [ 66%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_bgcolor PASSED [ 68%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_title PASSED [ 70%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_xlabel PASSED [ 72%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_xlim PASSED [ 74%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_ylabel PASSED [ 76%]
tests/test_scatter_plot_api.py::APITests::test_scatter_plot_no_ylim PASSED [ 78%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_XY_notequallength PASSED [ 80%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_X_notdatetime PASSED [ 82%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_Layout PASSED [ 84%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_bgcolor PASSED [ 86%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_num_std PASSED [ 88%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_period PASSED [ 90%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_title PASSED  [ 92%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_xlabel PASSED [ 94%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_xlim PASSED   [ 96%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_ylabel PASSED [ 98%]
tests/test_trend_plot_api.py::APITests::test_trend_plot_no_ylim PASSED   [100%]

- generated xml file: /home/cpv/workspace/cpv-python-local-build/services/tests/nosetests.xml -

----------- coverage: platform linux, python 3.7.6-final-0 -----------
Coverage XML written to file tests/coverage.xml

============================= 51 passed in 14.44s ==============================
==============Finished nose tests execution===============
==============Executing behave tests=============
1 feature passed, 0 failed, 0 skipped
12 scenarios passed, 0 failed, 0 skipped
48 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m1.463s
==============Finished nose tests execution=============
Quality Gates Passed for Project cpv_python.
Recording test results
[CucumberReport] Using Cucumber Reports version 5.3.1
[CucumberReport] JSON report directory is ""
[CucumberReport] Copied 1 json files from workspace "/home/cpv/workspace/cpv-python-local-build" to reports directory "/home/jenkins/.jenkins/jobs/cpv-python-local-build/builds/52/cucumber-html-reports/.cache"
[CucumberReport] Copied 0 properties files from workspace "/home/cpv/workspace/cpv-python-local-build" to reports directory "/home/jenkins/.jenkins/jobs/cpv-python-local-build/builds/52/cucumber-html-reports/.cache"
[CucumberReport] Processing 1 json files:
[CucumberReport] /home/jenkins/.jenkins/jobs/cpv-python-local-build/builds/52/cucumber-html-reports/.cache/services/Cucumber.json
Sending build status SUCCESSFUL for commit 7d1ac7b40f0c1e19ffb1264fa9b0dee9b4833db1 to BitBucket is done!
Finished: SUCCESS

The generated coverage.xml file:

<coverage branch-rate="0.6796" branches-covered="227" branches-valid="334" complexity="0" line-rate="0.7871" lines-covered="584" lines-valid="742" timestamp="1597851794196" version="5.1">
<!--
 Generated by coverage.py: https://coverage.readthedocs.io 
-->
<!--
 Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd 
-->
<sources>
<source>
/home/cpv/workspace/cpv-python-local-build/services
</source>
</sources>
<packages>
<package branch-rate="0.8333" complexity="0" line-rate="0.7941" name=".">
</package>
<package branch-rate="0.9936" complexity="0" line-rate="0.9691" name="plot">
</package>
<package branch-rate="0.3895" complexity="0" line-rate="0.5192" name="utility">
</package>
</packages>
</coverage>

I have tried referencing other similar posts in this community and elsewhere, but have had no luck in getting SonarQube to display/update the code coverage results.

Alexandre_Gigleux · August 21, 2020, 7:42am

Hello,

I’ve got the feeling you did not run your coverage.py command so that it generates details at file level so they can be loaded into SonarQube.

Here is an example of file content expected: https://github.com/SonarSource/sonar-python/blob/master/sonar-python-plugin/src/test/resources/org/sonar/plugins/python/coverage-reports/coverage.xml

Did you look at [Coverage & Test Data] Generate Reports for Apex, C/C++, Objective-C, Go, JS/TS and Python in the Python section and run the recommended commands?

Thanks
Alex

arajan · August 21, 2020, 2:28pm

Hi Alex,

Thanks for your guidance. I did generate the coverage report through pytest as shown in the code below.

I modified the code to follow the recommended logic, but unfortunately still no luck. Here’s my new code and the generated coverage.xml coverage.txt (31.4 KB) and nosetests.xml nosetests.txt (8.1 KB) .

coverage erase
coverage run -m nose2 
coverage xml -o tests/coverage.xml

pytest -o junit_family=xunit1 --junitxml=tests/nosetests.xml

I also have a .coveragerc file

[run]
branch=True
source = .
omit = /root/*, ./venv/*, *anaconda*

[report]
omit = /root/*, ./venv/*, *anaconda*, ./tests/*

Thanks
Abhi

Guillaume_Dequenne · August 27, 2020, 7:50am

Hello @arajan,

I had a quick look at the coverage.xml file you shared and I can see this:

<sources>
   <source>/home/cpv/workspace/cpv-python-local-build/services</source>
</sources>

By any chance, do you see something similar to the following lines in the logs of the Sonar Scanner?

WARN: Invalid directory path in 'source' element: /home/cpv/workspace/cpv-python-local-build/services
ERROR: Cannot resolve the file path '{filename}' of the coverage report, the file does not exist in all <source>.
ERROR: Cannot resolve {X} file paths, ignoring coverage measures for those files

If that’s the case,I believe that the issue lies with the fact that SonarQube can’t resolve the file paths mentioned in the coverage report. I suspect it is due to this part /home/cpv/workspace/cpv-python-local-build/ which seems unexpected given your properties. You’d probably have to correct the paths somehow.

See this post which describe a similar problem: Code coverage doesn't work with GitHub action

Hope that helps,
Guillaume

arajan · August 27, 2020, 3:42pm

Hi @Guillaume_Dequenne,

Thanks for your reply. It showed me the way to resolve the issue I was having. I had incorrectly checked the Prepare the SonarQube Scanner environment checkbox (even though it says not to). This meant that even though the paths were correct, the xml files were unavailable.

Thanks
Abhi

system · September 3, 2020, 3:42pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.