We self-host a SonarQube Enterprise 8.9.8 instance, and use Azure DevOps build pipelines with the latest marketplace Sonar extension to analyze our code. This includes the Prepare, Run Code Analysis, and Publish Quality Gate tasks. We see the issue below mainly on C# projects.
Analysis is run during both pull request and main branch builds. We are finding that sometimes when the PR does not have any Sonar issues, after the merge to main the analysis has more Sonar issues and fails its quality gate – the PR introduced Sonar issues into main. This seems contradictory to the Clean as You Code strategy. The quality gate on the PR succeeded, and we do not allow any PRs to merge to main without a successful PR quality gate. So I do not understand how the PR is introducing these issues into main.
We have tried changing the General setting Ignore duplication and coverage on small changes to false, as well as changing the shallow fetch value in our pipelines. Neither helped. I feel like this problem is related to the PR analysis being limited to new code, while the analysis on main is on the entire project. However, I cannot change the PR analysis to anything besides new code, so I have no proof this is the cause.
Can you characterize the kinds of issues that you see being ignored in your PRs and raised after merge? Are they, perhaps, new issues raised on old code? Because PRs only report issues on new code. Which means that, for instance, if you remove the only use of a variable, an ‘unused variable’ issue will be raised on the line where the var is declared. But if the variable wasn’t added/edited in the PR, that issue will be suppressed.
Yes, that is one of the issues. But there are issues of all sort, see below:
Unused variables
Duplicated blocks of code - one or two lines each. If Sonar is not analyzing the entire project for duplicate code, I can understand how this gets past the PR.
Code coverage not meeting the quality gate. Not sure how this is occurring since the coverage on new code should be the same or better than on existing code.
Too many nested control flow statements.
Need to merge enclosing if statements.
Roslyn errors.
It seems like at least, 3, 4 and 5 should have been caught in the PR but are getting leaked into master. 1 and 2 are still concerning, even if it is logical how they get into master. Is there any way to turn on full code analysis for PRs instead of just new code?
This global setting (Administration → General) is on by default. So if you have a 19-line PR with low coverage, it won’t fail the Quality Gate and could potentially drag down main after it’s merged.
It’s likely that all the rest (altho I’m not sure about #6) can be explained by changes in new code triggering new issues on old code.
