Pull request analysis clean, analysis after merge finds issues

Hi,

we are in the process of minimizing the SonarQube issues of our project. To reach this goal we want to be informed when new issues arrise at the time of CI-tests and code review. We integrated the SonarQube pull request analysis into our Jenkins jobs for this purpose.

Our problem: The pull request analysis is always clean, but after merging the changes the nightly build contains previously not found issues. In the pull request analysis the relevant source files and changes of the commit are correctly displayed.

Some info:

SonarQube 9.2.4 on premise, Scanner 4.6.2.2472-linux, Build Wrapper build-wrapper-linux-x86, Mercurial Extension

The project uses primarily C++ and we use Mercurial as our CVS. We configure the sonar-scanner as follows (with some information removed):

sonar.scanner.metadataFilePath=${WORKSPACE}/tmp/sonar-metadata.txt
sonar.pullrequest.key=[Jenkins build number, e.g. 3945]
sonar.pullrequest.branch=[URL to the code review system entry, e.g. https://rb.example.org/r/9594]
sonar.pullrequest.base=[Mercurial (and sonar) Branch on which the change will be merged, e.g. default]
sonar.login=[...]
sonar.host.url=[...]
sonar.projectKey=[...]
sonar.projectName=[...]
sonar.projectVersion=[...]
sonar.projectBaseDir=[...]
sonar.sources=src
sonar.sourceEncoding=UTF-8
sonar.language=cpp
sonar.c.file.suffixes=.c
sonar.cpp.file.suffixes=.cpp,.h
sonar.objc.file.suffixes=.m,.mm
sonar.cfamily.threads=4
sonar.cfamily.cache.enabled=true
sonar.cfamily.cache.path=@SONAR_CACHE_DIR@
sonar.cfamily.build-wrapper-output=@PROJECT_BINARY_DIR@
sonar.coverageReportPaths=@PROJECT_BINARY_DIR@/gcovr_sonarqube.xml
sonar.exclusions=src/external/**,utils/**,**/CMakeFiles/*,**/*.java
sonar.dependencyCheck.jsonReportPath=@PROJECT_BINARY_DIR@/dependency-check-report.json
sonar.dependencyCheck.htmlReportPath=@PROJECT_BINARY_DIR@/dependency-check-report.html

Maybe mercurial is the problem, but other projects are using mercurial and pull requests successfully (java projects using the maven sonar-scanner plugin, on the same SonarQube server with similiar settings for the pull requests).

Can you help me with this problem? Did somebody else report something similiar?

Best regards,

Jens Trillmann

Hello @jtrillma,

In order to better understand the issues you are encountering, we need additional information:

  • Can you provide us with some examples of issues that appeared in your nightly build?
  • In these examples, does the issue appear on lines of code that were modified as part of the pull request?
  • Is this something that appears often for you?
  • Are some rules more impacted by that than other rules?

Thank you!

Hi,

  • A partial list of rules that have found issues in the last 3 weeks:

    • cpp:S6230, Comparision operators ("<=>", "==") should be defaulted unless non-default behavior is required
    • cpp:S6007 [[nodiscard]] should be used when the return value of a function * should not be ignored
    • cpp:S1301 "switch" statements should have at least 3 "case" clauses
    • cpp:S3490 Special member function should not be defined unless a non standard behavior is required
  • All of the issues were found on lines that were present in their modified form in the pull request. But I found a difference between the not working project and the working project: the modified lines are not marked in the not working pull requests and the overview in the ‘Code’ tab states always ‘0’ lines. The files touched by the change are there, the lines are modified but they are not recognized as modified.

  • The pull requests don’t seem to work for the project at all, I could not find a single pull request indicating any issues.

  • Because I can’t find a pull request finding any issues I cannot say if some rules are impacted more than others.

I think the problem lies in the inability to recognize the modified lines as modified.

I would have assumed the ‘New Code’ settings would be the issue here, but they are the same between the different projects. I tried to choose a more fitting ‘New Code’ setting, but I have a conflict of interest: On the one hand we want the nightly build on the main branch to define New Code based on the previous analysis, but on the other we want the pull requests analysis New Code definition to use the difference between the main branch and the pull request.

Is there a parameter I can use with sonar scanner to define this behavior? Or am I mistaken and the issue lies elsewhere, maybe the detection of changed lines is faulty and New Code got nothing to do with it?

Best regards and thank you for your time,
Jens Trillmann

Hello @jtrillma, and thank you for this information.

I was thinking that only some issues were missing, but it indeed appears that the PR code is not detected at all. This is a different kind of issue, for which I won’t be able to help you much myself, but I’m pretty sure that if you post your scanner logs, other people will have a look at them.

Thank you regardless Loïc, here the requested log file. It is from a pull request analysis where previously fixed issues where reintroduced and should therefore be found (cpp:S5945).

false_negative_pull_request_analysis.log (65.7 KB)

The problem still persists. I tried keep the changes in the working directory (not committing them) and disabling the cache, without a change in behavior.

Here another log, but with debug output this time:

false_negative_pull_request_analysis_debug.log (743.3 KB)

I checked for the SCM reported changed lines for 0 files in the branch line in the log, but our maven project got the same message and displays the correct number of changed lines in the Sonar pull request analysis.

Can I provide you with more information? The issue still persists and I can’t figure out what the problem is.

Hi,

Thanks for the debug log. From that I can say (guess) that this looks is a question of identifying what’s new:

11:02:11.711 DEBUG: SCM reported changed lines for 0 files in the branch

You said earlier that new code detection is working for your other branches. Is there anything you can look at on the Mercurial side to see if the settings for the problem branch are different from the successful branches?

Also, going back to your OP, you say

To make sure I understand, you’re saying that the files changed in the PR show up as changed in SonarQube’s PR analysis, but with 0 issues in them?

Also, in the SonarQube UI, looking at one of these ‘surprise issues’ in the main branch, could you check the blame date on the issue line and see if it looks right?

 


And finally and tangentially (i.e. this has no bearing on your problem) I want to comment on some of your analysis configuration:

I suppose this is the default location & so no need to explicitly configure it here.

This seems odd to me, and guarantees that two analyses of the same PR won’t match up. I’m not familiar with Mercurial, but if it has a PR identifier, I suppose that’s what should be used here.

This is only needed under unusual circumstances. You should trigger analysis from the project base directory and then not need this parameter.

This does nothing

These are standard values. I would configure them globally in SonarQube and drop them from individual project analysis.

 
Ann

1 Like