ALM used (GitHub, Bitbucket Cloud, Azure DevOps) Azure DevOps
CI system used (Bitbucket Cloud, Azure DevOps, Travis CI, Circle CI Azure DevOps
Scanner command used when applicable (private details masked) Azure DevOps tasks: SonarCloudPrepare@1, SonarCloudAnalyze@1, SonarCloudPublish@1
Languages of the repository C#
My team uses SonarCloud integrated with Azure DevOps pipelines and SonarLint for Visual Studio. Both point at the same project(s) with the same underlying Quality Profile. However, we often notice that the code analysis results are different between SonarLint analysis in VS and pipeline analysis in DevOps.
A typical workflow is that developers make their changes in Visual Studio, run code analysis on the solution, fix detected issues, and create a PR in DevOps that triggers the validation build, including the SonarCloud analyzer. The problem is that SonarLint doesnât flag various issues, but they are flagged on PR builds. Please see the below examples.
Example #1 Note: This is the biggest problem as it generates for many files, sometimes resulting in dozens of auto-generated comments on the PR that have to be resolved manually.
SonarCloud analysis:
Code Smell: Missing XML comment for publicly visible type or member âREDACTEDâ (external_roslyn:CS1591)
SonarLint analysis:
No issue flagged
Example #2
SonarCloud analysis:
Code Smell: Remove this empty statement. (csharpsquid:S1116)
SonarLint analysis:
No issue flagged
Example #3
SonarCloud analysis:
Code Smell: Complete the task associated to this âTODOâ comment. (csharpsquid:S1135)
Itâs expected that this one isnât raised by SonarLint. The external_roslyn part of the rule key indicates that itâs not a Sonar rule. Itâs raised by some other part of your build and automatically imported into SonarCloud. (See the bottom of this page.)
Â
Thx,
Ann
P.S. Can you provide the version details of SonarLint and of Visual Studio, please?
Thanks for your response! That is correct, SonarLint is in connected mode bound to the same project that our pipeline in VisualStudio uses for the SonarCloud tasks. Thank you for the hint about external_roslyn. We will investigate this further. Of course the issues with other warnings are still outstanding as they seem to originate directly from Sonar.
Here are the software versions:
SonarLint 6.12.0.59751
Visual Studio Professional 2022 (64-bit) 17.4.2
@ernest_sioa are any Sonar/MS C# issues being reported in the IDE?
The Sonar C# and VB.NET rules are implemented as Roslyn analyzers, the same as the built-in Microsoft analysis rules. That means that Visual Studio controls when analysis is run and on which files.
By default, Visual Studio will not run analysis against the entire solution for performance reasons so you only see issues for a subset of files. See the Microsoft docs for more information.
Hi duncanp, thanks for your response! Indeed, we get most of the issues shown in DevOps reported in IDE as well. In Visual Studio, we manually run code analysis on a solution, and thatâs how we get the issues to appear.
However, as stated earlier, some of the issues (such as issues #2 and #3) donât appear in IDE, but they do appear in DevOps.
just to check, by âwe manually run code analysis on a solutionâ, you mean by selecting Analyze, Run Code Analysis, On Solution?
do the expected errors appear if you open the containing file?
The MS docs suggest that the menu command should run Roslyn analyzers (like those for Sonar C# and Sonar VB.NET).
However, I increased the MSBuild logging output in VS to detailed, clicked on the menu command, and looked at the output in the Build pane in the Output window. The /analyzer: arguments passed to csc.exe do not include any of the Sonar analyzers: it looks like they only contain the first-party Microsoft .NET analyzers.
So, it looks like Run Code Analysis does not run third-party analyzers by default, although the documentation does not say this. I havenât found a way of including the Sonar analysers in the set of analyzers to be executed.
Third-party analyzers like the Sonar ones are run live in the IDE when editing (depending on how you have VS configured - see custom analysis scope for more information).
So developers should be able to see (most*) Sonar issues while they are developing. Viewing all of the issues in the solution is overwhelming, so I normally have the Error List filter set to show only the issues for the current document:
*most: I say most Sonar issues should appear in the IDE because it appears there are certain kinds of Roslyn rule that are not executed in the IDE unless the analysis scope is set to the whole solution. Again, I havenât found this documented anywhere in the MS docs; itâs based on observation and experimentation. This limitation applies to rules that depend on the whole "compilation. However, most of our Roslyn rules donât require the whole compilation.
When you mentioned the Error List filter, I played about with it a bit, and it looks like the issue is not with the analyzer but instead the refreshing of that list. When I ran the code analysis on the solution (and yes, as per your question #1 this is indeed what we do: RMB click solution â Analyze and Code Cleanup â Run Code Analysis on Solution), the issues didnât initially appear. However, when I switched the filter to anything else, and then back âEntire Solutionâ I can see all issues properly listed.
I donât know if itâs an issue with the IDE, or the analyzer, but clearly, itâs a list refreshing problem. I checked on another developerâs machine and can confirm that we could reproduce the problem.
Thank you for that hint as it will help us detect all issues before we create PRs. Itâs still a bit inconvenient but it sounds like a Visual Studio problem that need to be flagged with Microsoft, unless you think that itâs related to Sonar analysis?
Just to comment on this one- thatâs not what we observe. We actually do get Sonar analysis results when running code analysis on a solution.
