Sonarqube Code Coverage Not Catching The Majority of Tested Code

SonarQube Server / Community Build
1
  • versions used
    Sonarqube Developer 8.9.2
  • error observed

When running a sonar scan on our code, some code that is covered by tests is not appearing as covered in the SonarQube analysis report. The example below is a PR that contains new code, and tests for that new code.

This is a method updateBMSCreditCheckResult() that is tested:

image
image1251×513 39.9 KB

After running mvn clean install, the JaCoCo report shows full code coverage on about 50% of the code.

However, the SonarQube analysis shows 0 coverage of this method:

image
image1055×621 38.7 KB

Here’s a test that that calls updateBMSCreditCheckResult():

    void givenValidInputs_whenUpdateBMSCreditCheckResult_thenReturnPassedCreditCheck() {
        testService.updateBMSCreditCheckResult(ACCOUNT_IDENTIFIER, BUSINESS_NAME, AddressFactory.createAddressResource());
        verify(billingRestClient, times(1)).remoteCallWrapped(any(),any(), any(), any(),any(),any());
    }
}

From my understanding, this test should at least cover the entry point of the method, and at least one of the conditionals.

Initially I thought that SonarQube was not reading any of our tests, but the SonarQube analysis report of this PR shows partial code coverage, meaning at least some of our tests are being recognized by SonarQube.

image
image1042×798 24.2 KB

Additionally, the tests themselves can be found under the SonarQube code section.

I recognize that SonarQube may have a different definition of coverage than JaCoCo, however from this post SonarQube and code coverage where Coverage = (CT + CF + LC)/(2*B + EL) I believe that a portion of our code qualifies as covered.

The code snippets I posted are just a single example, this is an issue we are encountering across our entire codebase.

Any help here would be greatly appreciated. Please let me know if this needs to be posted under a different section.

  • steps to reproduce
    We run SonarQube scans as a part of our Jenkins pipeline in a Kubernetes environment with the commands mvn clean install
    mvn sonar:sonar -Dsonar.login=myAuthenticationToken as per the documentation. We also pass the sonarqube configuration, branch type, and commit id as arguments.
2

Hello @swim224

As a reminder, from the documentation:

SonarQube doesn’t run your tests or generate reports. It only imports pre-generated reports.

It means that you have to provide the report generated from Jacoco to SonarQube (using sonar.coverage.jacoco.xmlReportPaths). Note that this property is pre-filled with common places where we expect this report to be placed, so it is possible that everything works without setting it explicitly.

Now, about the partial code coverage, I see two explanations:

  • The lines covered are from another language.
  • The analysis runs before the full report is generated.

At this point, you probably want to check the analysis logs, they will probably contain something related to the missing coverage.

Hope it gives you clues to continue the investigation.
Best,
Quentin

3

Hi Quentin,

Thanks a ton for the quick response. I’ll confirm the report is being passed to SonarQube properly. I assumed it was because we were getting some code coverage but given the results there definitely could be something strange going on there. I didn’t even consider that the report could be read part way through its generation. All our code is in Java so I’m leaning towards this being the likely explanation. I’ll investigate and let you know what I find :slight_smile:

Thanks!