SonarCloud, SonarLint, SonarCloud UI inconsistency

SonarCloud
1
  • ALM: Github
  • CI: Github Actions
  • Scanner Command:
  • Languages: C#
  • Error observed: No Errors

I have an issue that involves multiple facets of the sonarsource ecosystem. We’re running SonarLint and SonarScanner in our CI workflows in a GitHub Action. We have been trying to focus on cleaning up our code warnings and having a difficult time aligning all the different kinds of scans that are happening to where we get consistent results. Here’s the rundown of our setup:

CI in Github Actions… runs our build and sonarscanner. This results in warnings being returned from both the compiler as well as sonarcloud. When we run the scan on our main branch, we get warnings and if you load it into visual studio using the SonarLint extension you get warnings as well. The problem for us is we get different warnings at each level. What can we do to get more consistent results from these 3 scanners which in theory are doing the same thing. Here’s an example of the data we see…

CI: 333 Warnings
Visual Studio: 89 Warnings 116 Messages
SonarCloud UI: 44 “issues”

What contributes to the inconsistency here?

2

Hi,

Without knowing more about the warnings themselves, it’s difficult to know for sure.

I’m guessing the CI and Visual Studio are reporting compiler-level warnings that SonarCloud does not.

Can you provide more detail on the extra warnings in CI and VS? Can you also provide detail about your analysis configuration?

 
Thx,
Ann

3

That’s what was suspect to us too, but after reviewing we’re seeing compiler warnings in both places but then some additional “SXXXX” warnings that are being created by SonarCloud that aren’t being created by SonarLint. At first, we thought it was due to exclusions but we’ve removed the exclusions from the CI run and it made no difference to the end result. Could this have something to do with new code definitions? We’re running this particular job on our main branch nightly. Anything I can provide specifically to troubleshoot?

4

Hi,

Can you characterize what does/does not show up in each place?

 
Thx,
Ann

5

So here’s an example… in the CI run we see “S5332: Using http protocol is insecure. Use https instead.”. We don’t see this one at all via SonarLint or in the UI. Now I know this particular issue has been marked as “not going to fix” at some point. If that’s the case for why it doesn’t show up in SonarLint and the UI great, but why does it show up in the CI run? Is there a way to make it not show up in the CI run? Also, it looks like the excludes doesn’t work in the CI job. Here’s the command we’re using in CI:

dotnet-sonarscanner begin 
/d:sonar.qualitygate.wait=${{ inputs.sonar-quality-gate }}
/d:sonar.cs.opencover.reportsPaths="**/coverage.opencover.xml" 
/k:"${{ steps.sonarcloud-project-props.outputs.sonar_project_key }}" 
/o:"${{ steps.sonarcloud-project-props.outputs.sonar_org_key }}" 
/d:sonar.login="${{ inputs.sonar-token }}" 
/d:sonar.host.url="https://sonarcloud.io" 
/d:sonar.tests="**/*.Tests/" 
/d:sonar.exclusions="**/*.Tests/**/*,**/*.DataLoader/**/*" 
/d:sonar.coverage.exclusions="**/*.Tests/**/*,**/*.DataLoader/**/*,**/*.Data/**/*"

The one warning mentioned above is inside the DataLoader project that should be excluded.

6

Hi,

S5332 is a Security Hotspot rule. They aren’t executed in SonarLint.

What exactly do you mean that you see it in the CI but not in the UI?

Are you talking about the analysis logs? They’re not a good source for what is/is not raised during analysis. And BTW, Security Hotspots are presented in a different part of the interface. I suppose you’ve been looking for them among the Issues, which I suppose is why you think they’re missing…?

 
Ann

7

I’m aware of the hotspots and how they differ in their reporting in the UI. I didn’t realize that SonarLint ignored them.

So what shows up in Visual Studio via SonarLint then? Just code issues? When the scanner runs in CI, it’s reporting everything it finds (issues, security hotspots, etc…) and reporting those all as “warnings”?

8

Hi,

Is the focus the job logging?

Yes, SonarQube issues show up in those logs as warnings, because if they were reported as errors, they would break the build.

Why do they show up at all? Because they’re Roslyn checks & my understanding is that that’s just how Microsoft built it.

 
Ann

9

Well, the confusion is around what shows up in the log and what the UI/SonarLint reports. The log is showing everything regardless of whatever was marked as “not going to fix”. Then when we go to find those they’re not in Visual Studio either.

Thanks for talking through this stuff. It’s helpful to understand how these three tools report things. It’s frustrating they don’t all tell the same story though.

10

Hi,

Yep.

That’s how it works. Analysis re-finds everything, every time. Then the analysis report is bundled and sent to the server, and the server goes “Yep, I already knew about that one. It’s Won’t Fix”.

IMO you should just ignore the issues raised in the job log.

 
HTH,
Ann

11

Speaking of what the server knows, is there a way to “reset” those? Like what if at some point we decide we actually need to fix one of those things that’s marked that way?

12

Hi,

Yes, absolutely. You can change it back through the UI (and maybe the latest versions of SonarLint too.)

 
HTH,
Ann

13

Where do you do that?

14

Hi,

Go to the Issues tab of the project’s branch or PR.

 
HTH,
Ann

15

Ok I see where you can “undo” something like that. Now a slightly related question is, what determines whether or not something is scanned? If you look at our command to the sonarscanner we’re passing that filter. It appears that is being honored in the UI, but the scanner itself is still scanning it (we see stuff in the logs from the exclusions). Is there a way to exclude it from the logs as well?

16

Hi,

Sorry, what filter? I’m a bit lost. Is this about exclusions?

I’m begging you. Ignore the job log.

 
Ann