Pull request analysis clean, analysis after merge finds issues

Hi,

we are in the process of minimizing the SonarQube issues of our project. To reach this goal we want to be informed when new issues arrise at the time of CI-tests and code review. We integrated the SonarQube pull request analysis into our Jenkins jobs for this purpose.

Our problem: The pull request analysis is always clean, but after merging the changes the nightly build contains previously not found issues. In the pull request analysis the relevant source files and changes of the commit are correctly displayed.

Some info:

SonarQube 9.2.4 on premise, Scanner 4.6.2.2472-linux, Build Wrapper build-wrapper-linux-x86, Mercurial Extension

The project uses primarily C++ and we use Mercurial as our CVS. We configure the sonar-scanner as follows (with some information removed):

sonar.scanner.metadataFilePath=${WORKSPACE}/tmp/sonar-metadata.txt
sonar.pullrequest.key=[Jenkins build number, e.g. 3945]
sonar.pullrequest.branch=[URL to the code review system entry, e.g. https://rb.example.org/r/9594]
sonar.pullrequest.base=[Mercurial (and sonar) Branch on which the change will be merged, e.g. default]
sonar.login=[...]
sonar.host.url=[...]
sonar.projectKey=[...]
sonar.projectName=[...]
sonar.projectVersion=[...]
sonar.projectBaseDir=[...]
sonar.sources=src
sonar.sourceEncoding=UTF-8
sonar.language=cpp
sonar.c.file.suffixes=.c
sonar.cpp.file.suffixes=.cpp,.h
sonar.objc.file.suffixes=.m,.mm
sonar.cfamily.threads=4
sonar.cfamily.cache.enabled=true
sonar.cfamily.cache.path=@SONAR_CACHE_DIR@
sonar.cfamily.build-wrapper-output=@PROJECT_BINARY_DIR@
sonar.coverageReportPaths=@PROJECT_BINARY_DIR@/gcovr_sonarqube.xml
sonar.exclusions=src/external/**,utils/**,**/CMakeFiles/*,**/*.java
sonar.dependencyCheck.jsonReportPath=@PROJECT_BINARY_DIR@/dependency-check-report.json
sonar.dependencyCheck.htmlReportPath=@PROJECT_BINARY_DIR@/dependency-check-report.html

Maybe mercurial is the problem, but other projects are using mercurial and pull requests successfully (java projects using the maven sonar-scanner plugin, on the same SonarQube server with similiar settings for the pull requests).

Can you help me with this problem? Did somebody else report something similiar?

Best regards,

Jens Trillmann

Hello @jtrillma,

In order to better understand the issues you are encountering, we need additional information:

  • Can you provide us with some examples of issues that appeared in your nightly build?
  • In these examples, does the issue appear on lines of code that were modified as part of the pull request?
  • Is this something that appears often for you?
  • Are some rules more impacted by that than other rules?

Thank you!

Hi,

  • A partial list of rules that have found issues in the last 3 weeks:

    • cpp:S6230, Comparision operators ("<=>", "==") should be defaulted unless non-default behavior is required
    • cpp:S6007 [[nodiscard]] should be used when the return value of a function * should not be ignored
    • cpp:S1301 "switch" statements should have at least 3 "case" clauses
    • cpp:S3490 Special member function should not be defined unless a non standard behavior is required
  • All of the issues were found on lines that were present in their modified form in the pull request. But I found a difference between the not working project and the working project: the modified lines are not marked in the not working pull requests and the overview in the ‘Code’ tab states always ‘0’ lines. The files touched by the change are there, the lines are modified but they are not recognized as modified.

  • The pull requests don’t seem to work for the project at all, I could not find a single pull request indicating any issues.

  • Because I can’t find a pull request finding any issues I cannot say if some rules are impacted more than others.

I think the problem lies in the inability to recognize the modified lines as modified.

I would have assumed the ‘New Code’ settings would be the issue here, but they are the same between the different projects. I tried to choose a more fitting ‘New Code’ setting, but I have a conflict of interest: On the one hand we want the nightly build on the main branch to define New Code based on the previous analysis, but on the other we want the pull requests analysis New Code definition to use the difference between the main branch and the pull request.

Is there a parameter I can use with sonar scanner to define this behavior? Or am I mistaken and the issue lies elsewhere, maybe the detection of changed lines is faulty and New Code got nothing to do with it?

Best regards and thank you for your time,
Jens Trillmann

Hello @jtrillma, and thank you for this information.

I was thinking that only some issues were missing, but it indeed appears that the PR code is not detected at all. This is a different kind of issue, for which I won’t be able to help you much myself, but I’m pretty sure that if you post your scanner logs, other people will have a look at them.

Thank you regardless Loïc, here the requested log file. It is from a pull request analysis where previously fixed issues where reintroduced and should therefore be found (cpp:S5945).

false_negative_pull_request_analysis.log (65.7 KB)