I have an issue that involves multiple facets of the sonarsource ecosystem. We’re running SonarLint and SonarScanner in our CI workflows in a GitHub Action. We have been trying to focus on cleaning up our code warnings and having a difficult time aligning all the different kinds of scans that are happening to where we get consistent results. Here’s the rundown of our setup:
CI in Github Actions… runs our build and sonarscanner. This results in warnings being returned from both the compiler as well as sonarcloud. When we run the scan on our main branch, we get warnings and if you load it into visual studio using the SonarLint extension you get warnings as well. The problem for us is we get different warnings at each level. What can we do to get more consistent results from these 3 scanners which in theory are doing the same thing. Here’s an example of the data we see…
That’s what was suspect to us too, but after reviewing we’re seeing compiler warnings in both places but then some additional “SXXXX” warnings that are being created by SonarCloud that aren’t being created by SonarLint. At first, we thought it was due to exclusions but we’ve removed the exclusions from the CI run and it made no difference to the end result. Could this have something to do with new code definitions? We’re running this particular job on our main branch nightly. Anything I can provide specifically to troubleshoot?
So here’s an example… in the CI run we see “S5332: Using http protocol is insecure. Use https instead.”. We don’t see this one at all via SonarLint or in the UI. Now I know this particular issue has been marked as “not going to fix” at some point. If that’s the case for why it doesn’t show up in SonarLint and the UI great, but why does it show up in the CI run? Is there a way to make it not show up in the CI run? Also, it looks like the excludes doesn’t work in the CI job. Here’s the command we’re using in CI:
S5332 is a Security Hotspot rule. They aren’t executed in SonarLint.
What exactly do you mean that you see it in the CI but not in the UI?
Are you talking about the analysis logs? They’re not a good source for what is/is not raised during analysis. And BTW, Security Hotspots are presented in a different part of the interface. I suppose you’ve been looking for them among the Issues, which I suppose is why you think they’re missing…?
I’m aware of the hotspots and how they differ in their reporting in the UI. I didn’t realize that SonarLint ignored them.
So what shows up in Visual Studio via SonarLint then? Just code issues? When the scanner runs in CI, it’s reporting everything it finds (issues, security hotspots, etc…) and reporting those all as “warnings”?
Well, the confusion is around what shows up in the log and what the UI/SonarLint reports. The log is showing everything regardless of whatever was marked as “not going to fix”. Then when we go to find those they’re not in Visual Studio either.
Thanks for talking through this stuff. It’s helpful to understand how these three tools report things. It’s frustrating they don’t all tell the same story though.
Ok I see where you can “undo” something like that. Now a slightly related question is, what determines whether or not something is scanned? If you look at our command to the sonarscanner we’re passing that filter. It appears that is being honored in the UI, but the scanner itself is still scanning it (we see stuff in the logs from the exclusions). Is there a way to exclude it from the logs as well?