SonarQube does not detect the same issue in PR as on trunk

SonarQube Server / Community Build
, , ,
1

In our CI workflow we run SonarQube on Trunk and on each Pull Request branch.
We currently block a Pull Request if any issues are detected by SonarQube.

Sometimes we see that a green PR gets merged to trunk, and that SonarQube then reports reports new issues.
This results in a failed build chain. :disappointed:
We see this issue in our Java code.

It can happen in various cases, I will focus this post on the case where a method or variable becomes unused.

When SonarQube runs in context of the pull request, it does not catch that a class method or variable is no longer referenced, when the Pull Request is merged then the issue is detected and therefore fails on Trunk. :disappointed:

This even happens if the unused method/variable is in the same file as where the reference was removed from…

Information
SonarQube: 9.9 Enterprise
SonarQube Gradle plugin: 4.4.1.3373

The PR show up in SonarQube as PRs and the trunk is clearly marked as the “MAIN BRANCH”.

We set the following Properties:

  • sonar.projectKey
  • sonar.login
  • sonar.host.url
  • sonar.sourceEncoding = “UTF-8”
  • sonar.exclusions
  • sonar.cpd.exclusions
  • sonar.core.codeCoveragePlugin = “jacoco”
  • sonar.coverage.jacoco.xmlReportPaths

Additional for PR only:

  • sonar.qualitygate.wait = “true”
  • sonar.pullrequest.base = prTargetBranch
  • sonar.pullrequest.key = prBitbucketKey
  • sonar.pullrequest.branch = branchName

Please advice what could be wrong in our setup.

2

Hey there.

We are super aware of this issue (and it annoys us internally as well). Because we only raise issues on changed lines in PRs, some issues that get raised on unchanged lines (like. a variable no longer being used) don’t get raised until a full branch analysis. Luckily these rules are pretty few and far between.

You can vote on this roadmap item (New Pull Request issues on unchanged code).

3

Thank you for your response, I have now voted on the roadmap “idea”.

It seems very strange that you need to put this on the roadmap, it seems like unless you disable the rule then one of the bigger selling point for the paid version of SonarQube is broken.

It is strange to me how this is not a bigger issue for you.

4

The majority of rules aren’t affected here. Rules like “unused import” and “unused variable” are.

In exchange, we killed a lot of noise where issues were falsely being raised on PRs (trying to track issues with the target branch, which was possibly up to date, possibly not).

We want to improve the situation, and I expect we will, but sometimes we have to accept the tradeoffs we think make for the best experience.

5

Trade offs are part of the game.
Red builds are never nice, not for PRs not on Trunk.
Usually I would prefer a tradeoff where trunk is kept green to avoid affecting everybody.

If I understand your description correctly then the other issue could be solved by simply rebasing/merging the PR before running SonarQube, we already have automated that part.

I guess not using the PR feature and instead setting sonar.branch.name and sonar.newCode.referenceBranch would be a workaround?

6

Yes, that would make sure all issues, including those on unchanged lines or file-level issues, are raised.

I still think the value is low, given the few rules this issue affects. You would also lose PR decoration.