Scanner command: SonarCloudAnalyze@4 (with scannerMode: 'dotnet' in SonarCloudPrepare@4)
Languages of the repository: C#, XAML, XML, JSON, YAML, SQL
Only if the SonarCloud project is public, the URL: private
Error observed: See below
Steps to reproduce: See below
Potential workaround: Unknown
I have a problem with SonarQube Cloud analyzing my unit test projects. Most Sonar rules do not trigger/show up. It looks like only rules which explicitly only state “Test sources” in the “Analysis Scope” field on their documentation page do show up. Rules that state “Main and Test sources” do not show up.
Example:
I’ve added the following code block in a class both in my “main” code and in a test project.
[Fact] // Remove this line in the "main" code
public void MyTestMethod()
{
try
{
// TODO: Something
int i = 1;
//int i = 1;
}
catch { }
}
private abstract class Testing
{
public Testing()
{
}
}
If I then analyse this solution I get the following Sonar rule violation in my “main” code:
csharpsquid:S1135: Track uses of “TODO” tags
csharpsquid:S1481: Unused local variables should be removed
csharpsquid:S125: Sections of code should not be commented out
csharpsquid:S108: Nested blocks of code should not be left empty
csharpsquid:S1144: Unused private types or members should be removed
csharpsquid:S3442: “abstract” classes should not have “public” constructors
csharpsquid:S2486: Generic exceptions should not be ignored
Except for the last one (S2486), all of those rules state “Main and Test sources” in their documentation.
But for the test project only one different rule is reported:
csharpsquid:S2699: Tests should include assertions
I can already see this problem in the build step of my pipeline. “Main” code shows build warnings for rules like S125 or S1135. But test code doesn’t.
So it looks like the Sonar Roslyn analyzers are not correctly installed/enabled/configured for test projects.
Is this a bug?
If not: Is this a setting that I can change somewhere?
Indeed once a project is classified as Test project, only the rules for test scope will be activate for it.
What you can try to do is set <SonarQubeTestProject>false</SonarQubeTestProject> in the csproj of your test project (or each one of them). This should override the auto-classification mechanism.
If this is the case, then why does the documentation of many rules explicitly state “Main and Test sources” as “Analysis Scope”?
It does, but then the rule “csharpsquid:S2699: Tests should include assertions” is not shown. I do still want to see the rules that are specific for tests.
And this then also shows the rule “csharpsquid:S2486: Generic exceptions should not be ignored” in test code, which should explicitly only apply to “main” code according to its documentation.
Indeed, there is a gap here on documentation (I’ll make sure to update this). As you saw already, every rule in our .NET analyzer has a scope — MAIN, TEST, or ALL (Main + Test).
Main-classified project → gets MAIN + ALL scope rules.
Test-classified project → gets TEST scope rules — and, in your case, ALL-scope rules like S125 or S1135 are being left out of what’s actually uploaded to SonarQube Cloud, even though their docs say “Main and Test.”
On our side we already have a backlog ticket to fix this (internal access only). I don’t have an ETA, but I’ll update this thread once there’s progress.
This is the final behavior.
Something you could try in the meantime:
Since the project-classification ruleset is what’s limiting you, try enforcing the specific rules you want directly via .editorconfig, scoped to your test project paths.
Because .editorconfig severity settings are read by the Roslyn/SonarAnalyzer compiler pass directly, this can force those diagnostics to fire as build-time warnings for your test files independent of whether the project is classified as Main or Test.
edit: I just did a quick test and it seems like this problem also affects TypeScript code (analyzed via scannerMode: 'cli' in SonarCloudPrepare@4). Is your internal ticket specific to .NET analysis or for all languages?
This doesn’t seem to work. I’ve added these lines to an .editorconfig file:
I’ll investigate on my side the .editorconfig and come back to you.
Your editor config contains only the lines you pasted above?
I had an internal discussion to just to clarify the behavior and expectations on this topic.
For the .NET analyzer, the scopes are correctly applied on SQ IDE and in the case where you load the analyzer via NuGet, but they are not applied on CI analysis and that’s a conscious choice as far as I understood (I’ll gather some more info and come back on this).
So on our side (.NET analyzer) the scopes work as mentiond.
No, the .editorconfig file already existed and I just added those lines. So the file works/is found in general. The structure of the file is:
# NO 'root = true' here! This extends an .editorconfig file in the root folder.
[*.{cs,csx,vb,vbx}]
# Some rules like this one (works)
dotnet_diagnostic.CA1707.severity = none
# <snip>
# SonarQube rules for testing (doesn't work)
dotnet_diagnostic.S1135.severity = warning
dotnet_diagnostic.S1481.severity = warning
dotnet_diagnostic.S125.severity = warning
dotnet_diagnostic.S108.severity = warning
dotnet_diagnostic.S1144.severity = warning
dotnet_diagnostic.S3442.severity = warning
Please do. And if you missed my edit above, please note that this also seems to apply to other languages (like TypeScript).
I really don’t understand why I wouldn’t want to see the rules in test code. Especially for a simple rule like an remaining TODO comment.