Incorrect coverage report: unchanged lines of code in Java CI (Android/gradle)

We’ve integrated Sonar using GitHub Actions, but it reports 371 lines of code changing in the PR when 0 are modified.

https://sonarcloud.io/component_measures?id=hCaptcha_hcaptcha-android-sdk&metric=new_coverage&pullRequest=56&view=list

Hi,

Welcome to the community!

In fact, it shows 371 new lines to cover and 3.7k new lines total.

When I look, it appears that every line of every file (I didn’t check exhaustively) is marked as modified. Typically when we see this behavior it’s from automation somewhere in your process that changed whitespace. On this scale, it looks like line endings were probably changed (versus tabs <-> spaces since that would likely leave some lines untouched).

Can you take a closer look at your file histories & see if this is what happened?

 
Ann

You can see in the PR that no Java code at all changes.

I can only assume this is a bug on the sonar end. There have been no mass-reformat PRs.

Perhaps the issue comes only when a project is converted from “instant” to CI? It should be immaterial what transforms happen in the build process (in this case, none) given this was the first CI build.

Also, the sonar step completed before CI tests ran. Is it not doing an execution trace on the tests to compute coverage?

Hi,

On closer inspection, it seems that any random line I look at in any random file was “Committed on July 13, 2022”.

Is there anything unusual about your branch creation or PR analysis other than that you switched analysis methods?

And BTW, the PR that SonarCloud links to does actually show quite a number of code changes (altho not every line of every file).

 
Ann

You’re linking to a merge commit, which was one of the things we tried to see if the error was transient. The branch diff against main contains no Java changes.

What exactly are you looking at? That’s not what git blame says.

Not as far as we can tell.

Hi,

I’m not linking to a merge commit, SonarCloud is linking to it as the relevant commit for this PR/analysis. Can you do this again without the merge commit? They screw up the New Code detection.

I’m looking at SonarCloud
Selection_317

Yes, exactly. I’m trying to understand the difference.

You hit this problem in the process of switching to manual/full analysis, right? So let’s go back to that. Can you share your pipeline?

 
Ann

https://sonarcloud.io/summary/new_code?id=hCaptcha_hcaptcha-android-sdk&pullRequest=59

Looks identical, unfortunately.

It’s pretty straightforward: https://github.com/hCaptcha/hcaptcha-android-sdk/blob/f12b436a6b55f07c9d186fc728576531114436ef/.github/workflows/ci.yml

Hi,

Thanks for the pointer. I think I may have found the problem:
Selection_319

Can you try setting fetch-depth: 0? I think fetching just the 1 commit is what’s causing this bad commit date problem.

 
Ann

No change:

https://github.com/hCaptcha/hcaptcha-android-sdk/runs/7344011899

https://sonarcloud.io/summary/new_code?id=hCaptcha_hcaptcha-android-sdk&pullRequest=59

Hi,

Am I looking at the wrong run? I see --depth=1

And in the analysis log, I see:

Shallow clone detected, no blame information will be provided. You can convert to non-shallow with 'git fetch --unshallow'.
...
Missing blame information for the following files:
[long list]
...
This may lead to missing/broken features in SonarCloud

 
Ann

Looks like it needed to be moved up a line. That causes it to pass:

https://sonarcloud.io/summary/new_code?id=hCaptcha_hcaptcha-android-sdk&pullRequest=59

However, there’s still no coverage information.

Hi,

Now we’re at 0 new lines, presumably because there were no code changes in your PR:
Selection_321

SonarCloud’s only going to show coverage on New Code in a PR. No code → no coverage.

 
HTH,
Ann

OK, so in your estimation everything is now working as expected? I would have expected switching to CI to produce more results (of any type) vs. the instant scan.

Yes, I believe things are working as expected. To see more results… change some code. :smiley:

To be clear, on a full analysis (i.e. a branch analysis) you should see more thorough results. But on a PR, it’s only going to report results on what you changed. Since you didn’t change anything (code-wise) there’s nothing to report.

 
HTH,
Ann

Seems like there’s still an issue. Merging that change to main gives:

https://sonarcloud.io/component_measures?id=hCaptcha_hcaptcha-android-sdk&metric=new_coverage&view=list

With new spurious changes.