SonarScanner for .NET 8, now with other languages auto-detection

Clean Code Sonar Updates
,
1

Fellow .NET developers,
I would like to announce that the SonarScanner for .NET now supports multi-language analysis, starting with version 8.0!
This is one of the features you have been requesting to allow you to easily scan a whole repository containing both .NET and JS/TS projects, as well as IAC files, loose SQL etc.

What this means is that you only need a single scan to analyze your entire repository, including files from all of the following languages not referenced in the solution:

  • T-SQL
  • PL/SQL
  • JavaScript
  • TypeScript
  • YAML
  • XML
  • JSON
  • CSS
  • HTML

Furthermore, for JavaScript and TypeScript files, the scanner automatically detects test files based on the following patterns:

  • *.test.(sonar.javascript.file.suffixes + sonar.typescript.file.suffixes)
  • *.spec.(sonar.javascript.file.suffixes + sonar.typescript.file.suffixes)

This means you can now analyze a typical .NET backend+JS/TS frontend application with a single scan!

What happened before

Before version 8.0, the SonarScanner for .NET would analyze only the files that were part of the project. This meant that multi-language analysis was partially supported, and only for specific files explicitly included in the csproj, for example:

<ItemGroup>
<Content Include=“Included.js” />
</ItemGroup>

What happens now

Starting from SonarScanner for .NET v8.0, the files do not need to be part of the compilation. The only requirement for automatic multi-language analysis is that the begin, build, tests, and end scanner commands are run from the root directory of your solution.

For example, for a project that looks like this, the scanner should be run from the directory called “src”.

new way
new way960Ă—540 31.8 KB

Some things to note

  1. Since the Scanner is now auto-including all the files, the “Remove” node from .csproj files is no longer respected and files should be manually excluded via the sonar.exclusions parameter.
  2. Since this feature is turned on by default, there is also the option to opt-out manually, should you already have another analysis set up for the rest of the code using other means.To do this, you can specify “/d:sonar.scanner.scanAll=false” at the begin step. This will allow you to restore the previous behavior if you run hit your maximum LOC count.

Tricks

  1. Some of you had attempted to use custom XML rules to detect their own issues in csproj files. This should now work if you map the csproj (or vbproj) extension to the XML language.
  2. If you need to run tests on your JS/TS project you might run into problems since you are not in the proper directory for that. Please see Run npm script from another directory | Zander's Code Notes to run an NPM or Yarn script from a different directory.

As always, we welcome your feedback on this feature so we can make it as perfect as possible!
Now go write some clean code!

Denis

14 Likes
2

Why in the 9 hells do you make these breaking changes opt-out rather than opt-in?
Take a look at Windows Recall and the huge backlash Microsoft suffered from that. Should be an indication for how NOT to do things.

I’ve had an entire department blocked from working, because all tests suddenly go “Your LoC has increased 10-fold, forget it”

Furthermore I even went and counted our LoC using cloc (GitHub - AlDanial/cloc: cloc counts blank lines, comment lines, and physical lines of source code in many programming languages.) and it’s nowhere near the insane LoC SonarCloud reports.

Seems like a thinly veiled money grab to me, and we’re currently looking into replacing SonarCloud with another solution. I can only recommend others to do the same until you change practice. :rage:

1 Like
3

Hi @stsenvidan,

I am truly sorry this update had such a negative impact on your teams.

We made the decision to make this change as an opt-out after considering the alternatives. The previous behavior has been a long-standing sore point for many users, particularly newcomers. They could not figure out why our analysis did not find other files by default.
Had we made it an opt-in, most people would forever have to include this parameter to activate this new behavior, which is not the experience we want them to have.

We made sure to implement this change in a major version so that people could review it before adopting it. Our policy has always been to follow semantic versioning for this very purpose. It allows us to move the product forward and sometimes make major (or even breaking) changes when necessary.

Finally, we included a warning as part of the analysis indicating this feature is now active and how to remediate problems if any arise. We are updating it to be clearer on the issue. This warning will be removed when most people have migrated to the new scanner and are aware of the new behavior.

That being said, if your projects experienced an extreme increase in their LOCs count, I would like to know what type of files caused it exactly. Is it actual code we were not analyzing before (which is the whole point of this update), or are we catching files we should not be analyzing (in which case, this is a bug, and we need to know about it so we can correct it ASAP)? We are aware of some instances where we unfortunately analyze coverage files as XML which is an unwanted behavior, and we are investigating a fix for that.

If you are seeing things being analyzed that shouldn’t, we could open a new thread so we can take a look at what’s happening in your projects, if you do not mind sharing some information with us. That would help us quash bugs quickly.

Denis

4 Likes
4

Unfortunately I don’t know how to see what files Sonar is attempting to analyze, but the actual code LoCs in the repository is around 70k and of only a single language (C#), the reported LoCs by the analysis was 800+k.

If it’s prone to scanning code coverage xml files, that’s likely it as the codebase has a lot of testing.

As for the versioning, we’re using GitHub CI and it appears it just uses whatever the latest version is, thefore we didn’t know an update had happened, and as such weren’t aware of where to start looking.
The only thing we had to go by, was the final error in the analysis which just stated LoCs was too high. It took me quite some time and a lot of obsolete Stack Overflow questions etc. before I finally found a post on this community indicating the cause.

2 Likes
5

Thanks for your answer, @stsenvidan.

I would not say it is “prone”. It is another report we had, but I cannot seem to be able to reproduce the behavior yet.

Do you think you could send me the build logs from GH for a correct analysis and an erroneous one, as well as the build commands used in the workflow?
Maybe I can get some insight from that into what went wrong in your case.
This can, of course, be done over private messaging if you cannot post them publicly.

Denis

6

@stsenvidan we might have identified a case where we analyze coverage files. This seems to happen if the coverage files are generated outside of the test project directory. Could that be your case?

Denis

1 Like
7

Sorry about the lack of response - been hung up on tasks. But it sounds like you’ve found the culprit. Our coverage files end in up a coverage folder that is a sibling to the test folder (so they share a common parent folder)

Since this isn’t necessarily the same folder others might use, I think we’re stuck with the opt-out parameter to ensure our analysis sticks to just the C# code.

8

Thanks for confirming that it is indeed the source of your issue.
We are looking at how to best deal with that behavior right now.

Denis.

9

Hello @stsenvidan

I am sorry for the effect we caused on your organization.

To learn from this experience so that we can improve, I have a question about how you install the scanner. Do you install the dotnet tool in your GitHub action YAML (running dotnet tool install), or do you use a third party GitHub action from the marketplace?

10

We’re using our own runners for github (run on an Azure AKS), the dockerfile for own runners’ image installs the sonar tool using dotnet tool:

RUN dotnet tool update dotnet-sonarscanner --tool-path ~/.sonar/scanner

1 Like
14

Hello,
Very interesting feature, although when trying it out I saw the warning that when using sonar.excludes it might not work correctly.
If I use some exclude patterns I don’t get the JavaScript/TypeScript files at all, if I remove the flag I get them.
When do you expect to fix the behavior, allowing us to use excludes and detecting JS/TS files?
Thanks!

15

Hi @Vicentiuzor, welcome to the community!

The sonar.exclusions analysis property will be supported in v9 that should be released shortly.

Denis

16

We just released v8.0.2 of the Scanner for .NET that implements a fix for coverage files and enables the use of sonar.exclusions.

Please let us know if it works for your use-case.

17

Yes, using 8.0.2 seems to fix the sonar.exclusions issue and keeps identifying both .NET and JS/TS code. Thanks!

3 Likes
18

Hey, unfortunately, the property sonar.exclusions doesn’t seem to work in my project in version 8.0.2 or 9.0.0-rc. I had to downgrade to version 8.0.1 for it to work properly. I set the property in the project settings on SonarCloud. It’s based on .NET 9, I don’t know if this can cause any issue.

1 Like
19

Hi, @xC0dex

This seems strange. Let me check with the squad.
In the meantime, can you post a log of your analysis with v8.0.2 (redacted if need be) as text? Maybe run it with /d:sonar.verbose=true for good measure…

Denis

20

Sure! I’ve exported the log from the GitHub runner and uploaded it here as an attachment.
7_Analyze solution.txt (1.1 MB)

This is what I configured in SonarCloud:

image
image767Ă—147 16.6 KB

https://sonarcloud.io/code?id=xC0dex_APIWeaver

21

Thank you very much, @xC0dex!

Could you also tell me what you see that makes you say exclusions do not work in 8.0.2?

In the log I see the exclusions being detected. Do you see issues in SonarCloud that should not be there? Or duplications detected on excluded files?

Denis.

22

Sure!
In SonarCloud I would expect that, for instance, the demo folder is not visible in the Code tab and is also being ignored for the code coverage.

image
image1553Ă—309 15.5 KB

1 Like
23

Here is the log for 8.0.1.
7_Analyze solution.txt (1.1 MB)

One thing I noticed:
8.0.1:

sonar.host.url=https://sonarcloud.io
sonar.dotnet.excludeTestProjects=true
sonar.scanner.scanAll=true
sonar.cs.opencover.reportsPaths=**/test-results/**/coverage.opencover.xml
sonar.organization=apiweaver
sonar.visualstudio.enable=false

While 8.0.2 has the following log:

sonar.host.url=https://sonarcloud.io
sonar.dotnet.excludeTestProjects=true
sonar.scanner.scanAll=true
sonar.cs.opencover.reportsPaths=**/test-results/**/coverage.opencover.xml
sonar.exclusions=**/test-results/**/coverage.opencover.xml
sonar.organization=apiweaver
sonar.visualstudio.enable=false