Total coverage won't increase even after exclusions

[Shabar]

I have noticed the total coverage doesn’t increase even after exclude un covered lines.

What I have noticed after exclusions was, after exclusions the number of Lines to cover and Uncovered Lines both got reduced. So looks like that impact the total coverage remain same.

Further after exclusions those files DO NOT appear under files as expected which confirms exclusions working properly.

Can someone please explain what’s the theory behind and how exclusions can increase the total coverage

Before exclusions:

After exclusions:

[ganncamp (G Ann Campbell)]

Hi,

Welcome to the community!

On the face of it, what you’ve provided simply doesn’t sound right unless we’re dealing with rounding errors. Can you provide more details?

 
Ann

[Shabar]

Hi Ann,

Thanks for your response.

What sort of more details will help. As mentioned all the exclusions, has been removed from the Sonar side. I assume that implies data transfer from Project to Sonar is good.

Further does this can happen due to configuration issue on Sonar portal side?.

In theory to increase the Total coverage with eclusions, ONLY "Uncovered Lines" number should reduce after exclusions.

[Shabar]

In addition we use below command to publish to Sonar. Does that relates to above?

void publish_to_sonar() {
    withSonarQubeEnv('sonarq') {
        rtMavenRun(
                pom: './pom.xml',
                goals: "-B -Dapp.version=${APP_VERSION} -Djava.awt.headless=true org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -P sonar".toString(),
                resolverId: "APP_MAVEN_RESOLVER"
        )
    }
    echo "Published the unit test results to sonar"
}

[ganncamp (G Ann Campbell)]

Hi,

I see you’ve edited your OP to include some screenshots. That’s helpful. Can you also show before/after Lines of Code? Also, is it possible you excluded some code that was covered? That could certainly explain your coverage dropping.

And finally, your followup post includes your analysis command. Cool. I notice you’ve pinned in it the version of the analyzer which is less cool. You’re 2 major versions behind & this is why we advise against pinning the version. Drop that & it should automatically use the latest/greatest.

 
Ann

[Shabar]

@ganncamp

Drop that & it should automatically use the latest/greatest.

Can you please confirm the correct format to run above analysis command to run against the latest?

goals: "-B -Dapp.version=${APP_VERSION} -Djava.awt.headless=true org.sonarsource.scanner.maven:sonar-maven-plugin:LATEST:sonar -P sonar".toString(),

[ganncamp (G Ann Campbell)]

You should just be able to do sonar:sonar.

 
Ann

[Shabar]

Thanks Ann, that extracted the latest version during analysis.

To the initial query, I have checked covered classes and remove those from exclusions. However total percentage didn’t get increase as expected.

Further in the jenkins logs came across below warnings for most of the interfaces classes which haven’t excluded.

**21:12:14** [WARNING] Classes not found during the analysis : [..........]

Does that ring a bell?

I tried adding below to jenkins command, no luck though.

-Dsonar.java.binaries=target/classes

Note: This is multi module project

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\apache-maven-3.6.3\bin\..
Java version: 1.8.0_281, vendor: Oracle Corporation, runtime: C:\Java\jdk1.8.0_281\jre

[ganncamp (G Ann Campbell)]

Hi,

You are building before analysis, right?

Also, I think this question is still outstanding:

@ganncamp

Can you also show before/after Lines of Code? Also, is it possible you excluded some code that was covered? That could certainly explain your coverage dropping.

 
Ann

[Shabar]

@ganncamp

You are building before analysis, right?

Yes building before analysis.

@ganncamp

Can you also show before/after Lines of Code?

Can you please explain this bit further?

@ganncamp

Also, is it possible you excluded some code that was covered? That could certainly explain your coverage dropping

I checked this and it drops the coverage.

Any idea of [WARNING] Classes not found during the analysis .... , is it related?

[ganncamp (G Ann Campbell)]

Hi,

I should have been more precise. You’re rebuilding after the begin command and before the end command?

For Lines of Code, you’ve given screenshots of coverage before and after the exclusions. I’m looking for similar screenshots that show the Lines of Code metrics.

@Shabar

@ganncamp

Also, is it possible you excluded some code that was covered? That could certainly explain your coverage dropping

I checked this and it drops the coverage.

I’m not sure my question is answered. You excluded some code. Was any of the excluded code covered by tests?

 
Ann

[Shabar]

@ganncamp

I should have been more precise. You’re re building after the begin command and before the end command?

Sorry still not sure about what you want here? What do you mean by begin and end command?

@ganncamp

For Lines of Code, you’ve given screenshots of coverage before and after the exclusions. I’m looking for similar screenshots that show the Lines of Code metrics.

I don’t see option to get these metrics in the portal. (if you can direct me pl.)

@ganncamp

I’m not sure my question is answered. You excluded some code. Was any of the excluded code covered by tests?

Yes there are classes covered, but very less (below 15%),

[ganncamp (G Ann Campbell)]

Hi,

@Shabar

@ganncamp

I should have been more precise. You’re re building after the begin command and before the end command?

Sorry still not sure about what you want here? What do you mean by begin and end command?

Sorry, I was confused. I was somehow thinking this was a .NET project.

@Shabar

I don’t see option to get these metrics in the portal. (if you can direct me pl.)

This is what I’m looking for:

 
Ann

[Shabar]

@ganncamp

This is what I’m looking for:

[ganncamp (G Ann Campbell)]

Hi,

I was hoping for the same kind of before/after screenshots that you provided for coverage metrics. You see, I’m wondering about the math here. From your initial screenshots, I see that

Before exclusions you had

• 34,637 lines to cover
• 26,857 uncovered lines
• 7780 covered lines (via math)

After exclusions you had

• 31,808 lines to cover
• 25,169 uncovered lines
• 6639 covered lines (math)

So it seems that your exclusions dropped some covered lines. If I do a straight percentage (6639/31808*100) that comes out to 20.9%. But our coverage percentage isn’t based only on lines. It also takes into account conditions. I’ll let you review the formula but I think I’m back to my initial supposition: your exclusions eliminated enough covered code to drop the percentage.

 
Ann

[Shabar]

Appreciate your clarification Ann

Just one more thing I noticed, Changing below ${user.dir} to ${basedir} in jacoco plugin dropped the coverage percentage.

What’s the reason behind?

<plugins>
                    <plugin>
                        <groupId>org.jacoco</groupId>
                        <artifactId>jacoco-maven-plugin</artifactId>
                        <configuration>
                            <append>true</append>
                            <destFile>${user.dir}/target/jacoco.exec</destFile>
                            <dataFile>${user.dir}/target/jacoco.exec</dataFile>
                            <excludes>
                                  {some exclusions}
                             <excludes>
                                <outputDirectory>${user.dir}/target/coverage-reports/jacoco</outputDirectory>
                          </configuration>

[ganncamp (G Ann Campbell)]

Hi,

That’s really a JaCoCo question.

 

Ann