Ignored coverage

stechy1 · September 10, 2024, 12:21pm

I have a complex java project with many maven modules. Some of these modules are covered by unit tests. Coverage results from this tests are propagated to sonarcloud instance.
I also have special module for integration tests, where are performed more complex tests.
Next to this modules is another special module just for jacoco coverage aggregation.

My goal is to increase coverage by including results from integration tests module, but for some reason these tests are completely ignored.

The structure of the project is something like this:

parent
|- child1
|- child2
|- integration-tests
|- coverage-aggregation

Jacoco configuration for coverage-aggretagion module is following:

<!-- org.jacoco.jacoco-maven-plugin -->
<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <executions>

        <!-- Merge results from all coverage reports into single one -->
        <execution>
            <id>merge-results</id>
            <phase>verify</phase>
            <goals>
                <goal>merge</goal>
            </goals>
            <configuration>
                <fileSets>
                    <fileSet>
                        <directory>${project.build.directory}/../../</directory>
                        <includes>
                            <include>**/target/coverage-reports/jacoco.all.exec</include>
                        </includes>
                    </fileSet>
                </fileSets>
                <destFile>${project.build.directory}/coverage-reports/aggregate.exec</destFile>
            </configuration>
        </execution>

        <!-- Generate merged report from all modules -->
        <execution>
            <id>create-mega-merged-report</id>
            <phase>verify</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <formats>
                    <format>XML</format>
                </formats>
                <dataFile>${project.build.directory}/coverage-reports/aggregate.exec</dataFile>
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-mega-aggregate</outputDirectory>
            </configuration>
        </execution>

        <execution>
            <id>report-aggregate</id>
            <phase>verify</phase>
            <goals>
                <goal>report-aggregate</goal>
            </goals>

            <configuration>
                <formats>
                    <format>XML</format>
                </formats>
                <includeCurrentProject>true</includeCurrentProject>
                <dataFileIncludes>
                    <dataFileInclude>${project.build.directory}/coverage-reports/aggregate.exec</dataFileInclude>
                </dataFileIncludes>
            </configuration>
        </execution>

    </executions>
</plugin>

Does anyone had same issue or know how to force sonarcloud to not ignore results from aggregated results?

Thank you.

ganncamp · September 11, 2024, 12:06pm

Hi,

Welcome to the community!

Could you provide a full, debug analysis log, please?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

Thx,
Ann

stechy1 · September 11, 2024, 2:03pm

Please you can take a look into logs from sonar in the attached file.
The command used to create these logs is:

/home/petr/Dokumenty/mvnd/bin/mvnd org.sonarsource.scanner.maven:sonar-maven-plugin:sonar -X -Dsonar.branch.name=develop -Dsonar.branch.target=main > /tmp/sonar.log

sonar.log (10.3 MB)

ganncamp · September 11, 2024, 2:16pm

Hi,

This stands out to me:

[INFO] 15:55:53.425 Load/download plugins
[DEBUG] 15:55:53.446 Download plugin 'license' to '/home/petr/.sonar/_tmp/fileCache18133159224027425965.tmp'
[DEBUG] 15:55:53.447 --> GET https://scanner.sonarcloud.io/plugins/license/versions/7ce4b55d3b3f57a1119a281188e54928.jar
[DEBUG] 15:55:53.918 <-- 200 OK https://scanner.sonarcloud.io/plugins/license/versions/7ce4b55d3b3f57a1119a281188e54928.jar (470ms, 17405-byte body)
[DEBUG] 15:55:53.922 Download plugin 'text' to '/home/petr/.sonar/_tmp/fileCache2619409670578742892.tmp'
[DEBUG] 15:55:53.922 --> GET https://scanner.sonarcloud.io/plugins/text/versions/11a9b9e5d76fe507aaf1738f48e0b383.jar
[DEBUG] 15:55:59.428 <-- 200 OK https://scanner.sonarcloud.io/plugins/text/versions/11a9b9e5d76fe507aaf1738f48e0b383.jar (5505ms, 7022155-byte body)
[INFO] 15:56:00.315 Load/download plugins (done) | time=6890ms
[DEBUG] 15:56:00.316 Plugins not loaded because they are optional: [abap, sonarapex, csharp, cpp, cobol, dbd, dbdjavafrontend, dbdpythonfrontend, flex, go, web, jcl, javasymbolicexecution, java, javascript, kotlin, php, pli, plsql, python, rpg, ruby, sonarscala, swift, tsql, vbnet, vb, security, securitycsharpfrontend, securityjsfrontend, securityjavafrontend, securityphpfrontend, securitypythonfrontend, dart]
[DEBUG] 15:56:00.341 Plugins loaded:
[DEBUG] 15:56:00.341   * JaCoCo 1.3.0.1538 (jacoco)
[DEBUG] 15:56:00.341   * License for SonarLint 8.0.0.56531 (license)
[DEBUG] 15:56:00.341   * IaC Code Quality and Security 1.34.0.12019 (iac)
[DEBUG] 15:56:00.341   * Text Code Quality and Security 2.15.0.3845 (text)
[DEBUG] 15:56:00.341   * XML Code Quality and Security 2.10.0.4108 (xml)

The scanner now only downloads the language analyzers that are relevant to the project. The Java analyzer is not loaded, which means that the scanner didn’t find any Java files (i.e. any coverable files) in your project.

Other than the addition of the -X flag, did you run this job as you normally do?

Ann

stechy1 · September 11, 2024, 2:21pm

I did not used no more commands than I posted.
I can do it once again, but this time add “clean verify” before performing a sonar command.

ganncamp · September 11, 2024, 2:22pm

Hi,

Yes, please.

Ann

stechy1 · September 11, 2024, 2:24pm

Command is running. If it helps, here is a link to the project:
https://sonarcloud.io/summary/new_code?id=jfxsoft_jfxsoft.easymvc.parent&branch=develop

The coverage calculated in sonar is about 10%.
In IntelliJ I’m getting about 30%, because it take into an account results from integration tests.

stechy1 · September 11, 2024, 2:29pm

Please find new logs in the attached file.
sonar.log.zip (2.3 MB)

ganncamp · September 11, 2024, 3:41pm

Hi,

Thanks for the new log.

And I’m just realizing that you haven’t mentioned how your integration test results are produced. Are these also JaCoCo reports, or something else?

And are your reports in the coverage-aggregation module imported properly? Because going module to module in the log, I see that some reports are imported successfully, and some are not. E.G.:

[DEBUG] 16:23:54.357 Reading report '/home/petr/Dokumenty/jfxsoft/jfxsoft.easymvc.parent/jfxsoft.easymvc.procedure-activator/target/site/jacoco-aggregate/jacoco.xml'
...
[WARNING] 16:23:58.236 No coverage report can be found with sonar.coverage.jacoco.xmlReportPaths='/home/petr/Dokumenty/jfxsoft/jfxsoft.easymvc.parent/jfxsoft.easymvc/target/site/jacoco-aggregate/jacoco.xml'. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
...
[WARNING] 16:23:58.383 No coverage report can be found with sonar.coverage.jacoco.xmlReportPaths='/home/petr/Dokumenty/jfxsoft/jfxsoft.easymvc.parent/jfxsoft.easymvc.concurrent/target/site/jacoco-aggregate/jacoco.xml'. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
...
[WARNING] 16:23:58.623 No coverage report can be found with sonar.coverage.jacoco.xmlReportPaths='/home/petr/Dokumenty/jfxsoft/jfxsoft.easymvc.parent/jfxsoft.easymvc.config/target/site/jacoco-aggregate/jacoco.xml'. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
...

Ann

stechy1 · September 11, 2024, 4:03pm

Not all modules are covered with tests that is why coverage report is not found.
Yes, I’m using JaCoCo reports. Most interesting modules for you IMHO are:

jfxsoft.easymvc.it - here are performed integration tests
jfxsoft.easymvc.coverage - here should be aggregated coverage from all modules

When module has unit tests, results are sent to sonar and coverage is calculated. For example module “jfxsoft.easymvc.procedure” contains few unit tests and I can see some coverage in sonar.

ganncamp · September 11, 2024, 4:38pm

Hi,

For the integration tests, are those reports also in the JaCoCo format?

Ann

stechy1 · September 11, 2024, 4:39pm

Yes, everything is using JaCoCo

ganncamp · September 11, 2024, 4:40pm

Hi,

Then you’ll need to merge that into the other reports.

HTH,
Ann

stechy1 · September 11, 2024, 4:42pm

That is what I’m doing. In coverage module I have in the end one big result. When I analyze this result in IntelliJ, I get 30% coverage. The issue is that sonarcloud is ignoring this file.

stechy1 · September 11, 2024, 4:52pm

Every module inherits following configuration of JaCoCo configuration:

                <!-- org.jacoco.jacoco-maven-plugin -->
                <plugin>
                    <groupId>org.jacoco</groupId>
                    <artifactId>jacoco-maven-plugin</artifactId>
                    <version>${maven.jacoco.plugin.version}</version>

                    <executions>
                        <!-- Prepares the property pointing to the JaCoCo runtime agent which
                            is passed as VM argument when Maven the Surefire plugin is executed. -->
                        <execution>
                            <id>pre-unit-test</id>
                            <goals>
                                <goal>prepare-agent</goal>
                            </goals>
                            <configuration>
                                <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
                                <!-- Sets the name of the property containing the settings for JaCoCo
                                    runtime agent. -->
                                <propertyName>jacoco.agent.argLine</propertyName>
                                <append>true</append>
                            </configuration>
                        </execution>
                        <!-- Ensures that the code coverage report for unit tests is created
                            after unit tests have been run. -->
                        <execution>
                            <id>post-unit-test</id>
                            <phase>test</phase>
                            <goals>
                                <goal>report</goal>
                            </goals>
                            <configuration>
                                <!-- Sets the path to the file which contains the execution data. -->
                                <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
                                <!-- Sets the output directory for the code coverage report. -->
                                <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
                            </configuration>
                        </execution>

                        <execution>
                            <id>pre-integration-test</id>
                            <goals>
                                <goal>prepare-agent-integration</goal>
                            </goals>
                            <configuration>
                                <destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
                                <!-- Sets the name of the property containing the settings for JaCoCo
                                    runtime agent. -->
                                <propertyName>jacoco.agent.it.argLine</propertyName>
                                <append>true</append>
                            </configuration>
                        </execution>
                        <!-- Ensures that the code coverage report for integration tests is created
                            after unit tests have been run. -->
                        <execution>
                            <id>post-integration-test</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>report-integration</goal>
                            </goals>
                            <configuration>
                                <!-- Sets the path to the file which contains the execution data. -->
                                <dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
                                <!-- Sets the output directory for the code coverage report. -->
                                <outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
                            </configuration>
                        </execution>

                        <!-- Merge jacoco results -->
                        <execution>
                            <id>merge-unit-and-integration</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>merge</goal>
                            </goals>
                            <configuration>
                                <fileSets>
                                    <fileSet>
                                        <directory>${project.build.directory}/coverage-reports/</directory>
                                        <includes>
                                            <include>jacoco-ut.exec</include>
                                            <include>jacoco-it.exec</include>
                                        </includes>
                                    </fileSet>
                                </fileSets>
                                <destFile>${project.build.directory}/coverage-reports/jacoco.all.exec</destFile>
                            </configuration>
                        </execution>

                        <!-- Aggregate results -->
                        <execution>
                            <id>create-merged-report</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>report</goal>
                            </goals>
                            <configuration>
                                <dataFile>${project.build.directory}/coverage-reports/jacoco.all.exec</dataFile>
                                <outputDirectory>${project.reporting.outputDirectory}/jacoco-aggregate</outputDirectory>
                            </configuration>
                        </execution>

                    </executions>

                </plugin>

On top of this configuration is special configuration of “coverage” module which I have posted in my first post. As you can see I’m merging UT and IT into single report. Than in “coverage” module I’m merging report of each module into a single big coverage.

stechy1 · September 12, 2024, 7:54am

I can also provide logs from sonarcloud from background tasks…
sonarcloud.log (478.8 KB)

ganncamp · September 12, 2024, 12:43pm

Hi,

Thanks for the log, but the relevant parts are going to be CI-side.

Your analysis log shows coverage reports being picked up for some modules, including the first of the two that you explicitly called out for me.

How are you expecting analysis to find your consolidated coverage report?

Ann

stechy1 · September 12, 2024, 12:56pm

I have configured my Maven with properties where to find coverage analysis.

stechy1 · September 12, 2024, 12:58pm

And for example if I configure to read coverage only from my “coverage” modul, it will result with zero coverage.

ganncamp · September 12, 2024, 2:32pm

Hi,

What shows up in the log in this case? I suspect the problem is that the coverage report needs to be in the module, not in a ../somewhere-else module.

Ann

Topic		Replies	Views
Jacoco report imported but coverage not displayed in SonarCloud SonarQube Cloud java , coverage , jacoco	3	2045	November 7, 2022
Incomplete code Coverage in multimodule maven project SonarQube Server / Community Build sonarqube , maven , coverage	4	26	November 19, 2024
Sonarqube coverage for multimodule projects with integration tests SonarQube Server / Community Build coverage	1	245	May 31, 2024
Coverage reports are empty in Maven multi-module project SonarQube Cloud coverage , jacoco , multi-module	4	39	January 16, 2025
Multi module JAVA Maven project Jacoco code coverage report setup required for sonarqube (cover Unit and Integration Test report) SonarQube Server / Community Build coverage	6	3673	November 15, 2018

Ignored coverage

Related topics