How is New Code defined for new scans for multiple changes in the same Pull Request?

Environment:

  • Sonarqube 9.1 with Github
  • Using Github Flow where all PRs are targetting main branch
  • Using “Previous version” as New Code definition

I have been looking for hours and I’m still not clear on how New Code is defined in the context of PRs in the following scenario I have:

  1. PR with initial commit is submitted for scan via a CI pipeline by developer. Quality Gate fails due to 0% coverage on 10 new lines to cover.
  2. After first scan, more commits are done to the same PR with added coverage (although not for all lines that need it) and other fixes/changes.
  3. PR is again submitted for scan. Quality Gate fails again with 0% coverage, but now on 8 new lines to cover.

Why is coverage still 0% when coverage was added after it failed the Quality Gate the first time? I would expect it to increase based on the added coverage, even if it didn’t pass. There is definitely coverage added on the PR.

How does Sonar define new code in this situation where PR branch has multiple scans with different changes?

I thought Sonar could be considering new code after every scan, but this is clearly not the case as one of the new lines requiring coverage on the latest scan was submitted during the initial commit.
Unfortunately, Sonar doesn’t keep history of previous scans for the same PR so I cannot compare to see what it was complaining about on the first scan (and we unfortunately don’t remember).

As a related side note, what is the recommended way to set up New Code definition as for a project using Github Flow (merging everything to main branch). Would it be better to use “Reference Branch” (chose main branch as reference for the New Code)?

Hi,

Is it possible that subsequent commits factored out some of the new lines? If no, was there a rebase in there somewhere? We made improvements to the detection of new code (related to rebasing) early-ish in the 9-series, although not as early as 9.1. You should upgrade to the latest version, 9.8 (note that 9.9 LTS is due 7 Feb).

After that, for the question of 0% coverage, you should

  • make sure you’re generating a coverage report
  • feeding the report into analysis
  • and then check your analysis logs to make sure the report is being picked up and parsed properly by analysis

 
HTH,
Ann

Thank you for your reply @ganncamp! I will keep an eye to see if this happens again. We are for sure generating a coverage report and it’s being fed into the analysis.

Do you have thoughts/recommendations on my second question?
What is the recommended way to set up New Code definition as for a project using Github Flow (where there are no release branches and everything is merged to main branch)?

Would it be better to use “Reference Branch” (and chose main branch as reference) instead of “Previous Version” as definition for New Code?

Hi,

We try to keep it to one topic per thread. Otherwise it can get messy, fast.

We don’t have specific recommendations for Gitflow. If you’d like to dig into this, please create a new thread.

 
Ann

Makes sense! I will create a new thread on that to gather people’s thoughts. Thank you!

1 Like