SonarLint expected behaviour in VisualStudio

Hi,

We’re using Sonarcloud for analysis and have bound one of our solutions through the Sonarlint VS2017 extension. I noticed when using Sonarlint, I had no warnings appearing in the Error List for the solution, even when running a full code analysis.

From the Sonarlint output window, I could see that the solution was bound correctly, and was running for all projects apart from test projects, but no warnings were being given.

After spending a bit of time trying to find out what was wrong, in the end, it was solved by opening a file that had a pre-existing warning from the last analysis in Sonarcloud itself, and now warnings are appearing for all files.

It raises the following questions;

  1. Is this intentional? It seems like it wouldn’t be as it isn’t intuitive, or documented.

  2. Are test projects automatically excluded from analysis? Not a bad thing, just want to confirm.

  3. When exactly is the analysis run when opening a solution? Is it only when opening a file that has an issue, as seems to be the case from my experience noted above, or is this an issue?

  4. Since I have opened one file in a solution that contained a pre-existing error, opening other solutions does show the warnings for them even though they are not yet bound to Sonarcloud. Is it intended that Sonarlint can be ran standalone, or bound to Sonarcloud? (Want to make sure there isn’t an issue here either - this isn’t an issue as we’ll be binding all our solutions over time).

Thanks,
Adam.

2 Likes

Hi Adam,

(1 and 3) SonarLint has no say as to when the analysis is run - it’s entirely controlled by Visual Studio. There are a couple of VS settings that affect when analysis is run: firstly, enabling/disabling full solution analysis, and secondly, filters in the error list. However, VS will still try to be smart about running analysis incrementally in the background e.g. as soon as you start typing it will stop analysing. This does make it difficult to predict exactly when issues will appear, or whether you are seeing all of the issues that are going to be raised.

(2) Some rules are configured to run against all code, some only against non-test code, and a few only against test code. If you look at the definition for any rule in the online rule explorer you can see how each rule is configured (“Main sources” means “non-test code”).

(4) SonarLint can be either standalone or connected to a SonarCloud/SonarQube project.

Standalone mode: in standalone mode, you just install the VSIX and never connect to a Sonar server. VS will automatically run the SonarC# and SonarVB rules against any projects you open in the IDE (note: VS won’t run the rules as part of a build). You can configure the rules using normal VS/MSBuild rulesets.

Connected mode: there are two variants here, “legacy” connected mode and “new” connected mode. The difference is that in legacy connected mode the NuGet packages for the SonarC# and SonarVB analyzers are added to every project in the bound solution, which means the rules will be run on local builds. In “new” connected mode, the NuGet packages are not added, so the rules will only be run in the background in the IDE, and not in local builds.
Apart from that, the other features of the two variants of connected mode are the same: SonarLint will generate a ruleset from the Quality Profile (“QP”) for the connected Sonar project, so the rules run in the IDE match those in the QP.
In addition, any issues that have been suppressed in the connected project won’t be reported in the IDE, and finally, users of the Developer Edition of SonarQube will see notifications popping up in the IDE when the status of the Quality Gate changes.

Regards,
Duncan

1 Like

… so the rules run in the IDE match those in the QP.

Does it also match the severity level? Because I noticed that all the activated rules SonarLint get from the connected project are all shown as warnings even if a rule is “Critical” in the SonarQube project, it shows as warning in Visual Studio.

The severities are fixed currently. I thought there was already a ticket for this but I couldn’t find one, so I’ve created https://github.com/SonarSource/sonarlint-visualstudio/issues/808

@duncanp - thanks for the detailed response above, it was very useful. (and to Adam actually for asking exactly the same questions I wanted to ask!).

So the outstanding point seems to be visibility in the “Error List”, is that just a bug that the Error lists doesn’t reliably get populated? (Some of your answer hinted about reliability).

I’m having that problem at the moment, and we are using an on-premise SonarQube server. I also don’t want to tell the developers to “find a file with an existing issue so that you can see the full list of errors”.

Thanks

So - I think I have answered by own question.
When I go to the error list and select “Build+Intellisense” - then it populates the “Error List” pane. That does make sense, but obviously a “Build+SonarQube” would have been more intuitive.

@AubynC reliability: no, there isn’t a problem with the reliability of the Sonar rules. The uncertainty is about when they will be run as that is controlled by Visual Studio, taking in account the user-configurable VS settings mentioned above.

Build+Intellisense vs Build+SonarQube: we have no control over that part of the Error List; the behaviour is controlled entirely by VS.

Roslyn analyzers can be packaged either in a VSIX, or in NuGet packages (see the MS docs here).
If your projects reference Roslyn analyzers in NuGet packages then the issues will appear when the Build filter is active. If the Roslyn analyzers are packaged in a VSIX then the issues will appear when the “IntelliSense” filter is active.

SonarLint for Visual Studio packages the SonarC# and VB.NET analyzers in a VSIX so the issues will appear when the IntelliSense filter is active (although we do also ship our analyzers as NuGet packages, so if you have referenced our NuGet packages in your projects you will see Sxxx issues when the Build filter is active).

If all of that seems unnecessarily complicated and confusing, it is. Unfortunately, that’s the user experience provided by VS.

@duncanp - thanks for that additional breakdown, that provides extra insight.

I did suspect you were at the mercy of Visual Studio there. Whilst I do like the VS IDE there are multiple areas where the changes over time (to the VS IDE) that don’t always result in a smooth user journey.

Hi Duncan,
Can we apply Ruleset in standALone mode ? i am using SonarLint standAlone for c# project bu my ruleset is not getting applied to solution due to which my build is passing even if i have errors (S1128,1155,2971) configured in ruelset files.



@Gayatri please don’t add new questions to old threads - create a new thread instead.

Also, you’d already create a new post asking essentially the same question which I have already replied to. Please use that thread if you still need assistance.

This topic was automatically closed after 3 days. New replies are no longer allowed.