Quality Gate - Ignore small changes not being applied to branch analysis

[evan-masterson]

Hi,

We’re on the Developer Edition Version 9.9 (build 65466).

We’ve recently been looking at pull request analysis, as it has a more reliable analysis for direct merges rather than defining a reference branch to compare to.
However we’ve noticed an issue, or at least are unable to find any supporting documentation around the Quality Gate Ignore Small Changes configuration.

We have this setting turned on (default setting), which is what we desire, however it is only being applied or triggered during pull request analysis (where sonar.branch.name is not configured), and never by branch analysis, which results in false positive’s and failures occurring after merging.

See below the branch analysis in question, 7 lines of new code (below the threshold of 20), yet the quality gate fails, and the ignore small changes configuration is not being applied. (The info banner is not showing)

See below the pull request analysis in question, exact same branch, just 6 lines of new code (below the threshold of 20), now in this scenario the ignore small changes configuration is being applied as it’s intended purpose. (The info banner is showing)

The problem now arises where the quality gate has passed on our pull request, me merge our code, then main branch runs, and subsequently fails the quality gate in the same fashion as the branch analysis, due to the ignore small changes configuration not kicking in for the branch.

Should the ignore small changes configuration apply to all types of analysis? If yes, why is this not occurring in this scenario, is there another supporting setting or are we missing something else?
Have I misunderstood the ignore small changes configuration? Nothing in it mentions anything specific to only pull requests.

Can someone explain the purpose of this configuration in detail, and why it applies in one scenario and not another, as this is quite infuriating causing a misrepresentation of the analysis.

Regards,
Evan

[Ilham (Ilham Mayou)]

Hello Evan,

Thank you for your message and happy new year.

The setup to ignore duplications and coverage on small changes applies to both the 7 new lines in your PR and the 1.4k new lines that you have on your branch (see information under 0.0% Coverage and 0.0% Duplications). This is why the quality gate fails on your branch.

We have a list of enhancements planned regarding this for the future.

[evan-masterson]

Thanks for the reply,

So this is taking into account just new lines overall? Even on test files in headers, not on just new lines to cover, surely this setting should be based on just new lines to cover?
So it’s taking into account headers such as seen below where it’s changed from 2023 to 2024. Not even in source code.

But even more interesting is the fact that the branch analysis got run first, then the PR analysis got ran from the exact same git checkout, yet the new lines referenced in the Duplications section were wildly different, and the PR analysis only showed only the new lines of the file changed, when I can see on our runner that 1.4k files are modified from 2023->2024. So I would then expect the PR analysis to show 1.4k new lines.

Is there a difference between a branch analysis with a reference branch setup as main so the branch is being compared to main, vs the PR analysis for merge into main, are they analysed differently? As in this case it should be an identical comparison.

Are the enhancements you mention going to include a more in depth configuration to specify what should be considered ‘new’ code?

[Ilham (Ilham Mayou)]

Hello Ivan,

Thanks for your return.

Please find below my answers to your questions:

• We consider all new lines overall instead of the new lines to cover. I agree that this is a bit confusing as the display on the UI is different…This is part of the enhancements we are planning: [SONAR-11283] - Jira.

• The 1.4k new lines on your branch depend on the definition of new code that you have on your branch (since the last version, number of days, reference branch): From what I see, you defined the main branch as “reference branch” in your new code definition. Therefore, the 1.4k new lines are the result of the comparison between your branch and the reference branch.

• PRs display the difference between the new code added in the PR and the code in your target branch for merge. We analyze the new and updated code based on your DevOps PR definition.

• The results of the analysis of a branch using a reference branch as new code definition and the results of a PR analysis with the same branch code for merging into the reference branch are similar. There might be small differences due to the comparison algorithm we use to compare the code between the branch and the reference branch. For more details, see our documentation : https://docs.sonarsource.com/sonarqube/latest/project-administration/clean-as-you-code-settings/defining-new-code/#how-the-new-cod[…]nalysis-results

[evan-masterson]

Thank you for those additional answers.

I have 1 more question, see below here we made a version bump to main so no source code changes therefore our gate passes. This is a branch analysis, and main new code is based on previous version. Notice there are 0 new lines to cover and 0 new lines for duplication. This is a perfect example of when the ignore small changes configuration should kick in, but it’s not showing the ‘Some Quality Gate conditions on New Code were ignored because of the small number of New Lines’ banner here? Or because everything is 0 is that why it doesn’t show?

[Ilham (Ilham Mayou)]

Hello Evan,

Your analysis has no new lines. Therefore, no coverage and duplication calculations were made and the ignore setup has not been checked.

[evan-masterson]

Thanks for all the clarifications, they’ve helped a lot.

I’ve managed to get my initial test branch synced and the ignore small changes is correctly working on both the branch analysis and PR analysis.

I think this thread can be closed out
Thank you.