Sonarcloud old unit test code smell reappearing in new code analysis

[Chandrima (Chandrima Dasgupta)]

We use sonar cloud for our project code quality analysis. We are facing one problem very frequently now-a-days. When we check in any code, the unit test code analysis for the whole project is being triggered and coming as new code analysis outcome. E.g. I have abc.cs, def.cs, ghi.cs, xyzunittest.cs files in the branch. I updated only abc.cs file and checked in. I see the code analysis has new code smells in xyzunittest.cs. Did you hear about this issue before or have any clue why it might happen?

• Versions used (SonarQube, Scanner, Plugin, and any relevant extension)
  We use Sonarcloud. Azure DevOps Task (Scanner and Analysis) versions: 4
• Error observed (wrap logs/code around triple quote ``` for proper formatting)
• Steps to reproduce: Trigger one sonar cloud build
• Potential workaround: exclude the unit test files from analysis
• Scanner command used when applicable (private details masked)
  sonar.exclusions=/AssemblyInfo.cs
  sonar.test.exclusions=/AssemblyInfo.cs
  sonar.cs.vstest.reportsPaths=$(BUILD.SOURCESDIRECTORY)\Sources\TestResults**\TestResults*.trx
• In case of SonarCloud:
  • ALM used (Azure DevOps)
  • CI system used (Azure DevOps)

[Colin]

Hey there.

It’s a bit hard to understand the problem from your description. How about some screenshots, or a public sample project on SonarCloud that display the issue?

[Chandrima (Chandrima Dasgupta)]

Hi Colin,
Not sure if screenshot can really help. But check below image where there was no code smell.

Now we triggered a sonar cloud build after a new check in and it changed the code smells as below.

Now when I look into the code smells, the files were checked in long back and not changed in between the above 2 analysis

[Colin]

How about clicking into the 3 new bugs and sharing a screenshot of what issues were raised?

[Chandrima (Chandrima Dasgupta)]

here it is

[Colin]

Hey there.

I wonder if you were affected by this change:

DotNet: upcoming changes to how we detect test projects in Solution

Hello DotNet developers, Context We are about to make a change in how we detect “test” projects that are part of your Solution (.sln file). As you may already know, in SonarSource’s products, we make a difference between main projects and test projects. Test projects are only analyzed for issues related to unit tests and nothing more. We have realized the way we do it today for DotNet Solutions involves a few corner cases. Indeed, we guess that a project of your Solution contains only test fi…

Is your Unit Test Project (where xyzunittest.cs is located) being categorized as a Test Project? You can read more about what that means (and how to tell) here.

[Chandrima (Chandrima Dasgupta)]

Hi Collin,

It should be categorized as test project because the project name has ‘UnitTests’ in it. But can it cause reappearing of old code smells? Can it be solved by adding below line in the project itself? So we need to add it for all the unit test projects, right?
SonarQubeTestProject => true

Thanks!

[Chandrima (Chandrima Dasgupta)]

any updates for the issue?

[Chandrima (Chandrima Dasgupta)]

Not getting any updates for more than 2 weeks. Anyone working on it?

[Colin]

Hey there.

@Chandrima

It should be categorized as test project because the project name has ‘UnitTests’ in it.

The previously linked post indicates that this is no longer the criteria used to determine if a project is a test project.

@Alexandre_Gigleux

Indeed, we guess that a project of your Solution contains only test files by looking at its name. If the name contains “Test” or “Tests”, then it is automatically considered as a test project and the analyzer will run only the test rules in SonarLint (they are not analyzed at all on SonarQube or SonarCloud for now).

While this was convenient to avoid raising issues that will never be fixed on your test files, this is having a negative impact especially for new users discovering our products. Generally these new users create a simple fake solution made of bad code to evaluate our offering.

Because these users are just testing, they generally name their project “SonarTest”, “TestSonar” and as explained before, SonarCloud/SonarQube will raise nothing on them which would give the feeling that our products don’t detect any issue in the code.

Upcoming Change

We believe there is a better way to do this and we’re submitting it to you. In the coming weeks, a project will be considered as a “test” project only if it has some dependencies on test artifacts independently of its name.

It will be still possible to explicitly mark a project as being a test or main project by setting the MSBuild property <SonarQubeTestProject> to true or false e.g.

<PropertyGroup>
   <SonarQubeTestProject>true</SonarQubeTestProject>
</PropertyGroup>

The related documentation will be updated accordingly as soon as the change is effective.

Therefore, it’s worth checking how the project being categorized (looking at the analysis logs)

Please note this is a Community forum with volunteers offering help in using SonarSource produccts – this is not an issue tracking system.

[Chandrima (Chandrima Dasgupta)]

Hi Collin,

I was not aware that this is just a forum. We are a paid customer of SonarCloud and they asked us to post the query here. I was expecting some follow ups. Thanks for helping me out btw! I will try to get the logs.

Regards,
Chandrima