[Coverage & Test Data] Importing JaCoCo coverage report in XML format

Importing JaCoCo coverage report in XML format

Version 5.12 of our SonarJava analyzer deprecated uses JaCoCo’s binary format (.exec files) to import coverage. This binary format is internal to the JaCoCo project, and as such there are no guarantees for backward compatibility, so it should not be used for integration purposes.

As a replacement, when you scan a project with Sonar scanner, we use a special Jacoco report analyzer, which imports your JaCoCo’s XML coverage report, and this is the preferred option now. In this guide, I will describe how to import this XML report in some common scenarios.

You can find sample projects using the setup described here in this repository.

Maven

Remove any use of the sonar.jacoco.itReportPath, sonar.jacoco.reportPath, and sonar.jacoco.reportMissing.force.zero properties; they are deprecated and related functionality will be removed in the future.

We will use the jacoco-maven-plugin and its report goal to create a code coverage report. Usually, you would want to create a specific profile that executes unit tests with the JaCoCo agent and creates a coverage report. This profile would then only be activated if coverage information is requested (e.g., in the CI pipeline).

In the most basic case, we will need to execute two goals: jacoco:prepare-agent which creates the command line argument for JVM running the tests, and jacoco:report which uses data collected during unit test execution to generate a report in HTML, XML, or CSV format.

Here is an example of such a profile

<profile>
  <id>coverage</id>
  <build>
   <plugins>
    <plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.5</version>
    <executions>
      <execution>
        <id>prepare-agent</id>
        <goals>
         <goal>prepare-agent</goal>
        </goals>
      </execution>
      <execution>
        <id>report</id>
        <goals>
         <goal>report</goal>
        </goals>
      </execution>
    </executions>
    </plugin>
   </plugins>
  </build>
</profile>

By default the generated report will be saved under target/site/jacoco/jacoco.xml; this location will be checked automatically by the sonar-jacoco plugin so no further configuration is required. Just launch mvn sonar:sonar as usual and the report will be picked up.

If you need to change the directory where the report has been generated you can set the property either on the command line using maven’s -D switch

mvn -Dsonar.coverage.jacoco.xmlReportPaths=report1.xml,report2.xml sonar:sonar 

or you can set the property inside your pom.xml

    <properties>
        <sonar.coverage.jacoco.xmlReportPaths>../app-it/target/site/jacoco-aggregate/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>
    </properties>

Multi-module builds

With multi-module builds, we sometimes need to show coverage across modules. For example, we might have multiple modules implementing business logic and another module that contains integration tests for all these modules. We would like to see also coverage from this integration test module on business logic modules.

To achieve this, we can use JaCoCo’s report-aggregate goal, which will collect coverage information from all modules and create a single report with coverage for the whole project.

First, we still have to use prepare-agent goal as we did in the basic example in all modules, the best way to achieve this is to configure the execution of this goal in the parent pom.xml. Then we need to add execution to the module containing integration tests to generate the report across modules

<build>
  <plugins>
    <plugin>
      <groupId>org.jacoco</groupId>
      <artifactId>jacoco-maven-plugin</artifactId>
      <executions>
        <execution>
          <id>report</id>
          <goals>
            <goal>report-aggregate</goal>
          </goals>
          <phase>verify</phase>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

We snippet binds the goal to the verify phase. Now when we execute

mvn clean verify

we should find the aggregated report in target/site/jacoco-aggregate/jacoco.xml of the module containing the integration tests.

However, the JaCoCo plugin imports coverage reports module by module, so we need to import the same report multiple times (once for every module) to have coverage for all modules. To achieve this we set the property sonar.coverage.jacoco.xmlReportPaths in every module

<properties>
  <sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/../path_to_module_with_report/target/site/jacoco-aggregate/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>
</properties>

Troubleshooting

To investigate issues with the import of coverage information you can run Maven with the debug flag, -X:

mvn -X clean verify sonar:sonar 

In the logs, you will find the execution of different sensors for each module of the project. Typically you will have a log similar to the following one when the XML report is processed.

[INFO] 16:58:05.074 Sensor JaCoCo XML Report Importer [jacoco]
[DEBUG] 16:58:05.082 Reading report 'C:\projects\sonar-scanning-examples\maven-multimodule\tests\target\site\jacoco-aggregate\jacoco.xml'
[INFO] 16:58:05.093 Sensor JaCoCo XML Report Importer [jacoco] (done) | time=19ms

Gradle

Gradle includes the JaCoCo plugin in its default distribution. To apply the plugin to a project, you need to declare it in your build.gradle file together with the SonarScanner for Gradle.

plugins {
    id "jacoco"
    id "org.sonarqube" version "2.7.1"
}

The plugin provides the jacocoTestReport task, which needs to be configured to produce an XML report.

jacocoTestReport {
    reports {
        xml.enabled true
    }
}

By default, the report will be saved under the build/reports/jacoco directory and this location will be picked up automatically by the sonarqube plugin, so there is no further configuration required. To import coverage, launch

gradle build jacocoTestReport sonarqube

It is convenient to execute the jacocoTestReport task every time we execute test with the JaCoCo agent; to achieve this, we can run it after the test task. We can use finalizedBy to create a dependency between test and jacocoTestReport.

plugins.withType(JacocoPlugin) {
  tasks["test"].finalizedBy 'jacocoTestReport'
}
25 Likes

Gradle Multi-Module solution? Should we downgrade to SonarQube 7.9?

You cite a Maven multi-module solution to aggregate data across projects. With the old jacoco exec format, there is a Gradle multi-module solution with the jacocoMergeTest from JacocoMerge (part of gradle jacoco plugin). The Gradle JacocoMerge documentation doesn’t clearly list an XML data aggregation or merge.

The Version 8.* SonarQube no longer supports the exec data that jacocoMargeTest was aggregating for us. Does this mean we need to downgrade back to 7.9? We’ve lost Coverage tracking since we upgraded and I’m finding no solutions in my searches.

4 Likes

An update. It is looking like the gradle plugin is able to consolidate all the different module xml reports into the coverage data. Unfortunately, the coverage statistics went down by nearly half. So something is amiss. But at least it’s partially working. More investigation is needed here, and any insights from SonarSource would still be greatly appreciated.

2 Likes

Do you please know where could I find some actual and functioning guide on how to make gradle jacoco and sonarqube multi project build work with sonarqube 7.9 ? Thank you very much.

@Patrik_Polacek - if you are still using SonarQube 7.9, you should try the JacocoMerge solution described here: https://github.com/gradle/gradle/issues/8881#issuecomment-477600280

That is, if you are using Gradle. Not sure how with Maven. Good luck.

1 Like

For multi-module Maven builds, generate the comma-seperated list of xml files (one per module) dynamically by using the command

find . -path '*jacoco.xml' | sed 's/.*/${maven.multiModuleProjectDirectory}\/&/' | tr '\n' ','

and copy it to the sonar.coverage.jacoco.xmlReportPaths property. Alternatively pass the property in using the command line:

mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install org.jacoco:jacoco-maven-plugin:report

mvn sonar:sonar -Dsonar.coverage.jacoco.xmlReportPaths=$(find "$(pwd)" -path '*jacoco.xml' | sed 's/.*/&/' | tr '\n' ',')
7 Likes

Hi,

We believe it’s better if we don’t add messages in the “Guides” post. So if you have some question or addition, please create a new thread and if it will make sense our team will update the guide.

Thanks for the understanding and I’m locking the thread

3 Likes