SonarQube reports zero coverage from coverage.xml generated by coverage.py in Azure DevOps

SonarQube Server / Community Build
, ,
1

In my Azure DevOps build pipeline I use coverage.py to report coverage on Python code tested using unittest. I create a coverage.xml file in Cobertura format and use the PublishCodeCoverageResults@1 task to upload it. I see the expected coverage results in the Code Coverage tab of my build results page in the DevOps UI after coverage.xml.

However when I reference the same coverage.xml file using the sonar.python.coverage.reportPaths property in the subsequent SonarQubePrepare@4 task, I see 0.0% coverage reported for my project in the SonarQube server!

Looking at similar issues posted here (especially this one), I wonder if itā€™s because the filename entries in my coverage.xml use absolute paths?

Note that to get coverage.py to report coverage correctly I had to set its coverage run --source value to point at the python files installed under lib\site-packages on my DevOps VM. If I pointed at the checked-out source code instead I got zero coverage reported by coverage.py. I think thatā€™s because the tests are executing against the installed code rather than the source code.

Does anyone know how I can get SonarQube to show the coverage results in coverage.xml please?

2 Likes
2

Hi @cacti77

Would it be possible to share the Run Code Analysis task log in debug mode please ? There is certainly more details about the parsing of that coverage file.

Thanks !

3

@mickaelcaro Thanks very much for looking at this. Do you mean the log for the SonarQubeAnalyse task? Iā€™m afraid I canā€™t attach the whole log but if you can tell me which bits you might be interested in I might be able to give you those. Here are some extracts that might help you in the meantimeā€¦

	...
2020-07-29T14:34:46.1872553Z ##[section]Starting: SonarQubeAnalyze
2020-07-29T14:34:46.2097812Z ==============================================================================
2020-07-29T14:34:46.2099107Z Task         : Run Code Analysis
2020-07-29T14:34:46.2099404Z Description  : Run scanner and upload the results to the SonarQube server.
2020-07-29T14:34:46.2099639Z Version      : 4.11.0
2020-07-29T14:34:46.2099832Z Author       : sonarsource
2020-07-29T14:34:46.2100279Z Help         : Version: 4.11.0. This task is not needed for Maven and Gradle projects since the scanner should be run as part of the build.

[More Information](http://redirect.sonarsource.com/doc/install-configure-scanner-tfs-ts.html)
2020-07-29T14:34:46.2100805Z ==============================================================================
2020-07-29T14:34:46.7763499Z ##[debug]agent.TempDirectory=d:\a\_temp
2020-07-29T14:34:46.7798379Z ##[debug]loading inputs and endpoints
2020-07-29T14:34:46.7813061Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2020-07-29T14:34:46.7849731Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2020-07-29T14:34:46.7859214Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2020-07-29T14:34:46.7868299Z ##[debug]loading SECRET_SONARQUBE_ENDPOINT
2020-07-29T14:34:46.7875511Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2020-07-29T14:34:46.7885330Z ##[debug]loaded 5
2020-07-29T14:34:46.7911925Z ##[debug]Agent.ProxyUrl=undefined
2020-07-29T14:34:46.7916450Z ##[debug]Agent.CAInfo=undefined
2020-07-29T14:34:46.7918166Z ##[debug]Agent.ClientCert=undefined
2020-07-29T14:34:46.7918551Z ##[debug]Agent.SkipCertValidation=undefined
2020-07-29T14:34:46.9171466Z ##[debug]SONARQUBE_SCANNER_MODE=CLI
2020-07-29T14:34:46.9172300Z ##[debug]JAVA_HOME=C:\Program Files\Java\jdk8u262-b10
2020-07-29T14:34:46.9172677Z ##[debug]JAVA_HOME_11_X64=C:\Program Files\Java\jdk-11.0.8+10
2020-07-29T14:34:46.9173076Z ##[debug]JAVA_HOME_11_X64 path has been detected, switching to it for the analysis.
2020-07-29T14:34:46.9174920Z ##[debug]set JAVA_HOME=C:\Program Files\Java\jdk-11.0.8+10
2020-07-29T14:34:46.9177579Z ##[debug]Processed: ##vso[task.setvariable variable=JAVA_HOME;issecret=false;]C:\Program Files\Java\jdk-11.0.8+10
2020-07-29T14:34:46.9189358Z ##[debug]Absolute path for pathSegments: d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0,sonar-scanner,bin,sonar-scanner = d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner
2020-07-29T14:34:46.9194892Z ##[debug]which 'd:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat'
2020-07-29T14:34:46.9207117Z ##[debug]found: 'd:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat'
2020-07-29T14:34:46.9207731Z ##[debug]which 'd:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat'
2020-07-29T14:34:46.9212238Z ##[debug]found: 'd:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat'
2020-07-29T14:34:46.9213104Z ##[debug]system.debug=true
2020-07-29T14:34:46.9216613Z ##[debug]d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat arg: -X
2020-07-29T14:34:46.9220542Z ##[debug]d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat arg: --from=ScannerAzureDevOps/4.11.0
2020-07-29T14:34:46.9228125Z ##[debug]exec tool: d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat
2020-07-29T14:34:46.9228765Z ##[debug]arguments:
2020-07-29T14:34:46.9230110Z ##[debug]   -X
2020-07-29T14:34:46.9230852Z ##[debug]   --from=ScannerAzureDevOps/4.11.0
2020-07-29T14:34:46.9275783Z [command]C:\windows\system32\cmd.exe /D /S /C "d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat -X "--from=ScannerAzureDevOps/4.11.0""
2020-07-29T14:34:48.2439735Z 14:34:48.228 INFO: Scanner configuration file: d:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\..\conf\sonar-scanner.properties
2020-07-29T14:34:48.2504886Z 14:34:48.244 INFO: Project root configuration file: d:\a\1\s\sonar-project.properties
2020-07-29T14:34:48.3981370Z 14:34:48.397 INFO: SonarScanner 4.4.0.2170
2020-07-29T14:34:48.3982256Z 14:34:48.397 INFO: Java 11.0.8 AdoptOpenJDK (64-bit)
2020-07-29T14:34:48.3982768Z 14:34:48.397 INFO: Windows Server 2019 10.0 amd64
...
2020-07-29T14:34:57.6687577Z 14:34:57.668 DEBUG: Plugins:
2020-07-29T14:34:57.6689205Z 14:34:57.668 DEBUG:   * Cobertura 2.0 (cobertura)
2020-07-29T14:34:57.6691193Z 14:34:57.668 DEBUG:   * SonarCSS 1.2.0.1325 (cssfamily)
2020-07-29T14:34:57.6691928Z 14:34:57.668 DEBUG:   * Svn 1.9.0.1295 (scmsvn)
2020-07-29T14:34:57.6692887Z 14:34:57.668 DEBUG:   * SonarPLSQL 3.4.1.2576 (plsql)
2020-07-29T14:34:57.6693344Z 14:34:57.668 DEBUG:   * SonarScala 1.5.0.315 (sonarscala)
2020-07-29T14:34:57.6695938Z 14:34:57.669 DEBUG:   * C# Code Quality and Security 8.6.1.17183 (csharp)
2020-07-29T14:34:57.6696663Z 14:34:57.669 DEBUG:   * Vulnerability Analysis 7.9.0.5105 (security)
2020-07-29T14:34:57.6697234Z 14:34:57.669 DEBUG:   * Java Code Quality and Security 6.3.0.21585 (java)
2020-07-29T14:34:57.6698064Z 14:34:57.669 DEBUG:   * SonarHTML 3.2.0.2082 (web)
2020-07-29T14:34:57.6700906Z 14:34:57.669 DEBUG:   * SonarFlex 2.5.1.1831 (flex)
2020-07-29T14:34:57.6701465Z 14:34:57.669 DEBUG:   * SonarXML 2.0.1.2020 (xml)
2020-07-29T14:34:57.6701886Z 14:34:57.669 DEBUG:   * SonarTS 2.1.0.4359 (typescript)
2020-07-29T14:34:57.6703921Z 14:34:57.669 DEBUG:   * VB.NET Code Quality and Security 8.6.1.17183 (vbnet)
2020-07-29T14:34:57.6705191Z 14:34:57.670 DEBUG:   * SonarSwift 4.2.2.77 (swift)
2020-07-29T14:34:57.6706211Z 14:34:57.670 DEBUG:   * SonarCFamily 6.8.0.16475 (cpp)
2020-07-29T14:34:57.6707093Z 14:34:57.670 DEBUG:   * SonarPython 2.8.0.6204 (python)
2020-07-29T14:34:57.6708220Z 14:34:57.670 DEBUG:   * Code Smells 4.0.0 (smells)
2020-07-29T14:34:57.6708733Z 14:34:57.670 DEBUG:   * JaCoCo 1.1.0.898 (jacoco)
2020-07-29T14:34:57.6710253Z 14:34:57.670 DEBUG:   * SonarGo 1.6.0.719 (go)
2020-07-29T14:34:57.6710897Z 14:34:57.670 DEBUG:   * SonarKotlin 1.5.0.315 (kotlin)
2020-07-29T14:34:57.6737087Z 14:34:57.670 DEBUG:   * Azure Active Directory (AAD) Authentication Plug-in for SonarQube 1.2.0 (authaad)
2020-07-29T14:34:57.6738509Z 14:34:57.670 DEBUG:   * SonarTSQL 1.4.0.3334 (tsql)
2020-07-29T14:34:57.6739372Z 14:34:57.671 DEBUG:   * SonarJS 6.2.1.12157 (javascript)
2020-07-29T14:34:57.6740371Z 14:34:57.671 DEBUG:   * SonarRuby 1.5.0.315 (ruby)
2020-07-29T14:34:57.6741019Z 14:34:57.671 DEBUG:   * Vulnerability Rules for C# 7.9.0.5105 (securitycsharpfrontend)
2020-07-29T14:34:57.6741712Z 14:34:57.671 DEBUG:   * Vulnerability Rules for Java 7.9.0.5105 (securityjavafrontend)
2020-07-29T14:34:57.6742527Z 14:34:57.671 DEBUG:   * License for SonarLint 7.9.1 (license)
2020-07-29T14:34:57.6742954Z 14:34:57.671 DEBUG:   * Git 1.11.1.2008 (scmgit)
2020-07-29T14:34:57.6744304Z 14:34:57.671 DEBUG:   * SonarPHP 3.3.0.5166 (php)
2020-07-29T14:34:57.6744604Z 14:34:57.671 DEBUG:   * SonarABAP 3.8.0.2034 (abap)
2020-07-29T14:34:57.6744894Z 14:34:57.671 DEBUG:   * Vulnerability Rules for PHP 7.9.0.5105 (securityphpfrontend)
...
2020-07-29T14:35:11.3336014Z 14:35:11.329 INFO: Sensor Python Sensor [python]
2020-07-29T14:35:11.3355927Z 14:35:11.334 INFO: 1/1 source files have been analyzed
2020-07-29T14:35:12.2301792Z 14:35:12.227 INFO: Starting global symbols computation
2020-07-29T14:35:12.2302408Z 14:35:12.228 INFO: 149 source files to be analyzed
...
2020-07-29T14:35:37.7264190Z 14:35:37.725 INFO: Sensor Python Sensor [python] (done) | time=26396ms
2020-07-29T14:35:37.7265288Z 14:35:37.725 INFO: 149/149 source files have been analyzed
2020-07-29T14:35:37.7265968Z 14:35:37.725 INFO: Sensor Cobertura Sensor for Python coverage [python]
2020-07-29T14:35:37.7275139Z 14:35:37.726 DEBUG: Using pattern 'd:\a\1/tests/coverage.xml' to find reports
2020-07-29T14:35:37.8045916Z 14:35:37.803 INFO: Python test coverage
2020-07-29T14:35:37.8086476Z 14:35:37.807 INFO: Parsing report 'd:\a\1\tests\coverage.xml'
2020-07-29T14:35:37.8789570Z 14:35:37.877 INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=152ms
2020-07-29T14:35:37.8794698Z 14:35:37.877 INFO: Sensor PythonXUnitSensor [python]
2020-07-29T14:35:37.8809937Z 14:35:37.877 DEBUG: Using pattern 'xunit-reports/xunit-result-*.xml' to find reports
2020-07-29T14:35:37.9303375Z 14:35:37.929 DEBUG: No report was found for sonar.python.xunit.reportPath using default pattern xunit-reports/xunit-result-*.xml
2020-07-29T14:35:37.9304676Z 14:35:37.929 INFO: Sensor PythonXUnitSensor [python] (done) | time=52ms
...
2020-07-29T14:35:38.8518979Z 14:35:38.850 INFO: Analysis report generated in 267ms, dir size=5 MB
2020-07-29T14:35:39.4363669Z 14:35:39.435 INFO: Analysis report compressed in 584ms, zip size=1 MB
2020-07-29T14:35:39.4364382Z 14:35:39.435 INFO: Analysis report generated in d:\a\1\s\.scannerwork\scanner-report
2020-07-29T14:35:39.4364814Z 14:35:39.435 DEBUG: Upload report
2020-07-29T14:35:39.8930233Z 14:35:39.892 DEBUG: POST 200 https://[REDACTED] | time=457ms
2020-07-29T14:35:39.8947102Z 14:35:39.894 INFO: Analysis report uploaded in 459ms
2020-07-29T14:35:39.9064317Z 14:35:39.905 INFO: ANALYSIS SUCCESSFUL, you can browse https://[REDACTED]
2020-07-29T14:35:39.9065634Z 14:35:39.905 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
2020-07-29T14:35:39.9066227Z 14:35:39.905 INFO: More about the report processing at https://[REDACTED]
2020-07-29T14:35:39.9072415Z 14:35:39.906 DEBUG: Report metadata written to d:\a\_temp\sonar\20200729.5\c8ceb08d-aab0-ee62-ccd0-65b4d732db1a\report-task.txt
2020-07-29T14:35:39.9091671Z 14:35:39.908 DEBUG: Post-jobs : 
2020-07-29T14:35:39.9110036Z 14:35:39.910 INFO: Analysis total time: 42.169 s
2020-07-29T14:35:39.9194373Z 14:35:39.919 INFO: ------------------------------------------------------------------------
2020-07-29T14:35:39.9194848Z 14:35:39.919 INFO: EXECUTION SUCCESS
2020-07-29T14:35:39.9195300Z 14:35:39.919 INFO: ------------------------------------------------------------------------
2020-07-29T14:35:39.9195751Z 14:35:39.919 INFO: Total time: 51.835s
2020-07-29T14:35:40.0424324Z 14:35:40.042 INFO: Final Memory: 31M/110M
2020-07-29T14:35:40.0426177Z 14:35:40.042 INFO: ------------------------------------------------------------------------
2020-07-29T14:35:40.1182452Z ##[debug]Exit code 0 received from tool 'd:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat'
2020-07-29T14:35:40.1183729Z ##[debug]STDIO streams have closed for tool 'd:\a\_tasks\SonarQubeAnalyze_6d01813a-9589-4b15-8491-8164aeb38055\4.11.0\sonar-scanner\bin\sonar-scanner.bat'
2020-07-29T14:35:40.1201048Z ##[debug]Reverting JAVA_HOME to its initial path.
2020-07-29T14:35:40.1201683Z ##[debug]set JAVA_HOME=C:\Program Files\Java\jdk8u262-b10
2020-07-29T14:35:40.1206768Z ##[debug]Processed: ##vso[task.setvariable variable=JAVA_HOME;issecret=false;]C:\Program Files\Java\jdk8u262-b10
2020-07-29T14:35:40.1284376Z ##[section]Finishing: SonarQubeAnalyze
4

Is that this file that you gave in the property ?

Thanks !

5

Correct, yes. Hereā€™s how I configured the prepare task:

- task: SonarQubePrepare@4
  condition: and(succeeded(), ne(variables['skip.codescan'], true), eq(variables['python.version'], '3.7'))
  inputs:
	SonarQube: 'DSS-SonarQube'
	scannerMode: 'CLI'
	configMode: 'file'
	extraProperties: |
	  sonar.python.coverage.reportPaths=$(Agent.BuildDirectory)/tests/coverage.xml
7

And coverage.xml starts like this:

<?xml version="1.0" ?>
<coverage branch-rate="0.7016" branches-covered="2520" branches-valid="3592" complexity="0" line-rate="0.7622" lines-covered="5047" lines-valid="6622" timestamp="1596021179058" version="5.2">
	<!-- Generated by coverage.py: https://coverage.readthedocs.io -->
	<!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
	<sources>
		<source></source>
	</sources>
	<packages>
		<package branch-rate="1" complexity="0" line-rate="1" name="C:.hostedtoolcache.windows.Python.3.7.8.x64.Lib.site-packages.redacted">
			<classes>
				<class branch-rate="1" complexity="0" filename="C:/hostedtoolcache/windows/Python/3.7.8/x64/Lib/site-packages/redacted/__init__.py" line-rate="1" name="__init__.py">
					<methods/>
					<lines/>
				</class>
			</classes>
		</package>
8

Following the advice here I added this line in a bash task immediately after the task in which I create my combined coverage.xml:

sed -i 's/C:\/hostedtoolcache\/windows\/Python\/3\.7\.8\/x64\/Lib\/site-packages/MySource/g' coverage.xml

In my sonar-project.properties file I have this (MySource is just made up here):

sonar.sources=MySource

Then SonarQube was able to show my coverage results as expected, and these of course matched the view in the Code Coverage tab of the build summary UI in DevOps.

So it seems the file paths in coverage.xml must be discoverable under the sonar.sources folder. The problem is that SonarQube is analysing my checked-out source code but Iā€™m running the actual unit tests and coverage.py against the installed code. I want to do the latter because I believe itā€™s a more realistic scenario for end-users, plus I have to install my python package anyway because itā€™s using C++ extensions. In other words, running the tests from the Python source code alone simply wouldnā€™t work.

I suspect that if coverage.py and SonarQube were both pointed at the checked-out code then this zero coverage issue in SonarQube wouldnā€™t happen. But I canā€™t do that.

If anyone knows of a better workaround than hacking coverage.xml for my use-case that would be greatly appreciated:).

10

could you expand a bit on how you resolved this?

I am facing this issue and I am modifying the coverage result to the correct path in my build server as far as I can tell, but I am still facing an issue.

I have posted here about it:

Thanks

11

Hi - it looks like youā€™ve resolved or worked around your issue now. Did you still want to know anything? In my case I used sed to edit the paths in coverageā€™s report file so that SonarQube could find those paths under the directory given as the value for sonar.sources in the sonar-project.properties file. Otherwise SonarQube would report zero coverage.

12

Yes thanks I was able to resolve in the end. I had to use the absolute path on the build server taken, rather than something like ā€˜./ā€™.

Thanks for replying.

13

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.