How to explain differences between Jacoco report and Sonarqube dashboard?

Hi,
I’m on Sonarqube 8.9 LTS and I’m trying to have jacoco coverage report shown in Sonarqube for my multi-module project.

So far I’ve followed your instructions and it seems to work : [Coverage & Test Data] Importing JaCoCo coverage report in XML format

But when I compare the jacoco report to what is shown in Sonarqube, it is very different. I have no exclusions for sonarqube.

Can you help me understand the differences ?

Jacoco report :

Sonarqube corresponding display :

As you can see, the lines count, and coverage don’t match at all. What should I fix to get corresponding numbers ?

P.S. : if it’s helpful, infra and domain both have unit tests and infra depends on domain (hexagonal architecture).

Hi,

There are a few things to be aware of here. First, while SonarQube takes your coverage report as gospel in terms of which lines in a file are/aren’t covered, if your coverage report omits a file… SonarQube counts it anyway. This (slightly dusty) blog explains.

The second thing is that SonarQube is going to make its own count of Lines of Code. So it’s quite unsurprising to me that the “lines” numbers don’t match between the two sources. Beyond that, there’s Executable Lines to take into account. There is a difference between Lines of Code and Executable Lines (I think this is exposed to view in the UI as “Lines to Cover” in the Measures tab). So this is why none of the math matches: only the “covered lines” would be the same between the two.

Does that help?

 
Ann

Thanks for your answer Ann, but I’m still quite confused. Let me ask a more precise question : There are tests that covers instructions for my “infra” module as shown in jacoco report, and 27% of instructions are covered.

How can it be calculated/displayed as “0%” coverage in Sonarqube ?

Hi,

Can you click through on infra & see if all the files/directories in it are listed at 0% too? This may be a rounding issue. If not, then we need to go back to the analysis log to make sure the coverage report is being picked up.

 
Ann

Sure, so I’ve check a particular file which is 100% covered in Jacoco local report. In Sonarqube, it is shown with 0% coverage, so I guess there was some problem during importing.

When I activate debug logs, I see no warnings, no errors (see below)

Here is my conf in various modules and parent :

maven parent/aggregator :
<sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>

infra :
<sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/../jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>

domain :
<sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/../jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>

jacoco-report-aggregate :
<sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/target/site/jacoco-aggregate/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>

Here is the resulting debug logs :

[INFO] 09:36:54.727 Sensor JaCoCo XML Report Importer [jacoco]
[INFO] 09:36:54.730 Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.
[DEBUG] 09:36:54.730 Reading report '/var/lib/jenkins/workspace/sonarqube-analysis-code-coverage/domain/../jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml'
[INFO] 09:36:54.828 Sensor JaCoCo XML Report Importer [jacoco] (done) | time=101ms

[INFO] 09:36:55.034 Sensor JaCoCo XML Report Importer [jacoco]
[INFO] 09:36:55.035 Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.
[DEBUG] 09:36:55.035 Reading report '/var/lib/jenkins/workspace/sonarqube-analysis-code-coverage/jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml'
[INFO] 09:36:55.077 Sensor JaCoCo XML Report Importer [jacoco] (done) | time=43ms

[INFO] 09:37:19.557 Sensor JaCoCo XML Report Importer [jacoco]
[INFO] 09:37:19.558 Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.
[DEBUG] 09:37:19.558 Reading report '/var/lib/jenkins/workspace/sonarqube-analysis-code-coverage/infra/../jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml'
[INFO] 09:37:19.603 Sensor JaCoCo XML Report Importer [jacoco] (done) | time=46ms

[INFO] 09:37:19.803 Sensor JaCoCo XML Report Importer [jacoco]
[INFO] 09:37:19.804 Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.
[DEBUG] 09:37:19.804 Reading report '/var/lib/jenkins/workspace/sonarqube-analysis-code-coverage/jacoco-report-aggregate/target/site/jacoco-aggregate/jacoco.xml'
[INFO] 09:37:19.819 Sensor JaCoCo XML Report Importer [jacoco] (done) | time=16ms

Do you see a problem in this conf, or do you think it could be a Sonarqube bug ?
Also my “infra” module is packaging as war, it’s not a problem for jacoco, but can it mess with Sonarqube recalculations ?

Hi,

I’m not seeing anything wrong with either your properties (altho theoretically you shouldn’t have to specify this for Maven) or your logs. I would like to see fuller logs though. Could you post the whole thing, please?

 
Ann

Nope, I can’t unfortunately :slight_smile:

But I’m working non stop on this from yesterday and now you confirm the “import” part looks good, I’m digging into the report generation. I think the report is correct locally but some parameter or profile activation is messing up the jacoco report on Jenkins.

Hi,

Do you see any log lines about missing or unrecognized files / paths?

Also, feel free to redact as necessary. :slightly_smiling_face:

 
Ann

Oh yes, I can’t agree more, but your jacoco import plugin needs this to work sadly :

" the JaCoCo plugin imports coverage reports module by module, so we need to import the same report multiple times (once for every module) to have coverage for all modules. To achieve this we set the property sonar.coverage.jacoco.xmlReportPaths in every module"

source : [Coverage & Test Data] Importing JaCoCo coverage report in XML format