Coverage % is different on CircleCI and SonarCloud

SonarQube Cloud
,
1
  • CI system used: CircleCI
  • Languages of the repository: typescript/javascript, that’s a GCP Cloud Function
    The problem is that the Line Coverage % and coverage % overall differ between local/CircleCI test runs, and the ones that SonarCloud shows inside the “Coverage” section of the project. Here’s the screenshot from the SonarCloud:
    Screenshot 2023-08-30 at 12.03.44
    Screenshot 2023-08-30 at 12.03.441126×749 39.6 KB

    And here is the coverage report log from the CircleCI, which is shown before the SonarCloud job run:
------------|---------|----------|---------|---------|---------------------------------------------------------
File        | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s                                       
------------|---------|----------|---------|---------|---------------------------------------------------------
All files   |   74.82 |    60.46 |   55.55 |   75.17 |                                                         
 helpers.ts |      50 |    27.27 |      50 |   48.71 | 58-76,86-91,103-115                                     
 index.ts   |   84.11 |    65.33 |   66.66 |    84.9 | 121,149-150,164,168,172,177,185,190,195,223,229,275-280 
------------|---------|----------|---------|---------|---------------------------------------------------------
Test Suites: 1 passed, 1 total
Tests:       7 passed, 7 total
Snapshots:   0 total
Time:        5.305 s, estimated 11 s

As you may see here is the difference between “% Lines” - “SonarCloud Line Coverage” from the log and the screenshot:

All Files: 75.17% - 75.8%
helper.ts: 48.71% - 47.5%
index.ts: 84.9% - 85.8%

The difference isn’t that big, because of the small codebase scale. But on a larger scale, it is much greater, sometimes getting over 5-10% at least, which is the difference between passing or not 80% minimum threshold.

Can someone tell me why that might be happening and if it is normal for SonarCloud to show slightly different results for the same report file?

2

Hey there.

Two avenues to pursue:

  • It looks like you’re performing a Pull Request Analysis (or analysis of a short-lived branch). Keep in mind that only new lines (added or modified) or included in the calculation here. If these are preexisting files, not all lines will be considered as New Lines to Cover. You can see which lines are considered “new” when you click on the file.

  • If these are 100% new files, you should be able to track down specific discrepancies between SonarCloud and the raw coverage report. Once you click into the files you have coverage data on the left side of the code viewer. Ultimately the math SonarCloud is displaying is right: (153-37)/153 = 75.8%. So if these are 100% new files, the discrepancy has to be in either the number of lines to cover, or the uncovered lines.

3

Hi Colin!
Thanks for the clarification.
Is there a way to check not coverage for the new code, but overall coverage, as it is in the Jest coverage report?

4

You get that when you analyze a long-lived branch (most commonly your main branch). You can also see a preview of the coverage measure on the pull request overview (Estimated after merge %), but keep in mind that is a metric that combines condition/line coverage as defined here.