which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
- SonarQube 10.4
- CFamily plugin version: 6.52.1.66618
I’ve a Jenkins job that runs a sonar scan for my project and I have seen for the past year irregular reports of issues. For example if you look at the screenshot below, the amount of issues reported for 2nd Analysis is 0, however when you click on the “See full history of analyses”, there it says otherwise and shows the amount of issues found was 7.
That is one of the issues I have seen over the year and we would like some sort of information on why such matters occur.
In addition, from the same Jenkins job mentioned above, if a run an analysis, and right after the analysis is done, I run the same job again, the results of the 2 analyses are different although no code changes/commits have been made. See below, it is the scan that I ran after the one at 2.19 pm from the images above.
That first screenshot isn’t showing the number of issues. It’s showing the number of added issues. That’s why there’s a + in front of that 0.
Are you only re-running the analysis? Or are you re-running the full pipeline, including the build and build-wrapper file generation (or compilation database)?
I see, but the issue is that:
10.31 AM scan returned 7 issues
11.26 AM scan returned 0 issues
1.24PM returned 7 issues
2.19PM returned 7 issues
3.22 PM returned 0 issues
Should’t the 1.24 PM scan show +7 issues as the issues went from 0 to 7?
Full pipeline which includes building the project with the c++ buildwrapper. Note that we run the analysis using make -j {certain number} in other words the analysis is being executed in parallel, is parallel runs an issues?
Full command: /path/to/build-wrapper-linux-x86/build-wrapper-linux-x86-64 --out-dir <output directory> make -j ${jobs_number}
Fair enough. I already knew (and reported internally) that this was broken for negative changes (issues removed). I’ll augment my report with this too. And beyond that, let’s set this aside for this thread. We try to keep it to one topic per thread. Otherwise it can get messy, fast.
Ehm… Is the analysis running in parallel or the build? We automatically parallelizeanalysis, so that’s not a problem.
There are some known causes for indeterminism in the CFamily analyzer. The two usual suspects are:
[CPP-5067] Fix analysis results consistency between runs when a file has multiple configurations in one BW/CD
[CPP-5514] Nondeterminism-bug in the symbolic execution engine can cause flaky issues
The former was fixed in the CFamily analyzer 6.56, available starting with SonarQube 10.6.
The latter hasn’t been fixed yet.
If you are compiling the same source file for multiple targets as part of your build (i.e., compile for both 32 and 64 bits), the first one is the most likely (depending on the build and analysis order, you get the issues for either one of the two architectures, not both, and the rule may trigger depending on, say, the size of an integer or a pointer).
If the flaky rule has the tag “symbolic-execution,” then it is the second problem.
Please, let me know if this helps identify the source of the flakiness.
Sorry, I should have been more explicit. In the SonarQube interface, the issue that flickers, it has a rule ID, and a set of tags. It looks like this
I am interested mainly in the rule ID (S2107 in this case), but if you see the tag I have underlined in red, then it is most likely the issue CPP-5514.
Hi @aalvarez ,
I went through our bugs and code smells but no sign of ID S2107 or symbolic-execution. In addition to that, the code is compiled in 64 bits only.
Then, I would need the scanner’s logs (in debug mode) and the output of build-wrapper to see if I can find some hints there. Would you be able to send them to me via private message?
