Sonarqube only detects issues on some pipeline builds, on some pipeline builds issues are detected, on others, issues are not detected.
When issues are not detected, reports: “0 issues, quality gate passed” (like there are no issues, only there are definitely issues still present on changed lines of code)
Once the PR (with outstanding undetected issues) is merged, the issues do usually appear in the main branch, but they are unassigned!
Another thing is the analysis graph which jumps up and down an unreasonable ammount (the code changes alone do not account for this)
The issues coincide with a few changes (but cant be exact as they are intermittent):
Update to devops server
addition of a git submodule in the codebase
change to it infrastructue
Another thing i noticed is this, it only shows up on the main/master branch (not on any PR branch build!) and it only shows up intermittently :
I have compared the log outputs from both a successful scan that detected issues, and a scan that did not find issues (that is should have detected), i saw no change.
Apologies for sparse information, i dont have too much time to create this post and dont want to share anything im not supposed to
Do you have debug-level scanner logs from both a “good” and a “bad” scan? It would be really interesting to compare debug-level logs to see if both runs are seeing the same files and running the analyzers correctly on them. We can get an approximation of this by having a look at the “Lines of code” metric in the Activity tab (where you saw the Issue count jumping up and down). What we need to understand is if the fluctuation in issues is caused by a fluctuation in the files/lines of code analyzed, or if the analyzed code stays reasonably stable.
Another place where you could look for hints is in the projects’s Background tasks > 3 dots next to a background task > Show scanner context. This will show you the full configuration that was used for a given scanner run. If you can find a good and a bad scan and correlate them with their corresponding background tasks, perhaps you can spot if the difference in issues found is caused by a difference in configuration. To correlate a scan run with its background task, you can check the logs: at the very end, upon submitting the results to the server, it typically prints the background task ID.
Just focusing on the request for the lines of code metric: It seems that the “Lines to cover” is the fluctuating issue. There are some big jumps in the Lines which are expected:
first one was changing the build type used for the CI build
second one was accidentally adding the thirdparty dependencies to the scan
third one was restoring the sonar exclusions to ignore the thirdparty folder
Hey @Rory, thanks for checking that. “Lines to cover” is indeed a good measure to check because it typically corresponds with main code, which is where most issues are raised. It’s interesting to see that the pattern doesn’t strictly correlate with those 3 main events you outlined, instead it correlates with the fluctuations in issues. So it could totally be the case that the fluctuation in issues is caused by a fluctuation in lines/files analyzed.
I suggest you start collecting debug-level scanner logs and, whenever you experience another fluctuation, compare the logs and also perhaps check out the scanner context for both runs.
We could help check out debug-level logs, there might be some technical details in them for us. If you manage to get debug logs for 2 very different runs, let me know. We may ask you to share them via private message.
Hi @Rory, what you did enabled debug-level server-side logs. What we need is debug logs from the Scanner in your CI, which is enabled with a flag passed to the Scanner command. The exact way to do it might depend on how you are running the scanner but the main ways to do it are either:
Pass -X to the scanner command, or
Set the sonar.verbose=true property, either in a sonar-project.properties file, or via command line like -Dsonar.verbose=true.
I had a quick first look at the logs. Here are some interesting points:
The two analyses do not seem to analyze the same stuff, if you search for the following strings in both logs, you’ll see different numbers:
Number of compilation units in the provided Compilation Database (949 vs 586)
Number of remaining compilation units to be analyzed (513 vs 340)
compilation units were fully analyzed (561 vs 375)
So I think that the issue is not in the analysis itself, but in the way the compilation database is created.
The log tells it is generated by the build wrapper. One classical hypothesis is that it is based on incremental build instead of a full build, which lead to an incomplete compilation database, and therefore an incomplete analysis.
We would need to see the compilation database files and to know how they are generated if further investigation is needed.
