Project files are indexed as test files

Must-share information (formatted with Markdown):
Version information:

Enterprise Edition Version 7.6 (build 21501)
SonarQube task 4.6.3
TFS 2017 U2 

Hey folks!
I’m having an issue with implementing SonarQube build task in TFS build definition for a certain project. The main problem is that all sources files are considered to be test files after running msbuild UNLESS you run vstests task after it’s completion.

GUID.sonar.tests=\  #huge list of all the files in the project

In the Run Analysis task:

    .cs' generated metadata as test  with charset 'UTF-8'
    .cs' indexed as test with language 'cs'

I’m quite unsure why is this the case. I’ve checked project definition for Test project type GUID - none is present.

I noticed this in MSBuild task:

    2019-08-13T14:51:05.2856481Z                    Task "GetAnalyzerSettings" (TaskId:31)
    2019-08-13T14:51:05.2856644Z                      Task Parameter:AnalysisConfigDir=d:\agents\agent02-DEV\_work\17\.sonarqube\conf (TaskId:31)
    2019-08-13T14:51:05.2856782Z                      Task Parameter:Language=cs (TaskId:31)
    2019-08-13T14:51:05.2856957Z                      Task Parameter:CurrentProjectDirectoryPath=d:\agents\agent02-DEV\_work\17\s\AgentPortalWeb (TaskId:31)
    2019-08-13T14:51:05.2857083Z                      Task Parameter:
    2019-08-13T14:51:05.2857198Z                          OriginalAnalyzers=
    2019-08-13T14:51:05.2857348Z                              ..\packages\Roslynator.Analyzers.2.0.0\analyzers\dotnet\cs\Roslynator.Common.dll
    2019-08-13T14:51:05.2857526Z                              ..\packages\Roslynator.Analyzers.2.0.0\analyzers\dotnet\cs\Roslynator.Common.Workspaces.dll
    2019-08-13T14:51:05.2857698Z                              ..\packages\Roslynator.Analyzers.2.0.0\analyzers\dotnet\cs\Roslynator.CSharp.Analyzers.CodeFixes.dll
    2019-08-13T14:51:05.2857873Z                              ..\packages\Roslynator.Analyzers.2.0.0\analyzers\dotnet\cs\Roslynator.CSharp.Analyzers.dll
    2019-08-13T14:51:05.2858029Z                              ..\packages\Roslynator.Analyzers.2.0.0\analyzers\dotnet\cs\Roslynator.CSharp.dll
    2019-08-13T14:51:05.2858216Z                              ..\packages\Roslynator.Analyzers.2.0.0\analyzers\dotnet\cs\Roslynator.CSharp.Workspaces.dll (TaskId:31)
    2019-08-13T14:51:05.2858514Z                      Task Parameter:OriginalRulesetFilePath=C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Team Tools\Static Analysis Tools\\Rule Sets\MinimumRecommendedRules.ruleset (TaskId:31)
    2019-08-13T14:51:05.2858719Z                      Task Parameter:ProjectSpecificConfigDirectory=d:\agents\agent02-DEV\_work\17\.sonarqube\conf\0 (TaskId:31)
    2019-08-13T14:51:05.2858863Z                      Task Parameter:IsTestProject=True (TaskId:31)
    2019-08-13T14:51:05.2859044Z                      Reading config file: d:\agents\agent02-DEV\_work\17\.sonarqube\conf\SonarQubeAnalysisConfig.xml ... (TaskId:31)
    2019-08-13T14:51:05.2859228Z                      Commencing retry-able operation. Max wait (milliseconds): 2500, pause between tries (milliseconds): 499 (TaskId:31)
    2019-08-13T14:51:05.2859833Z                      Operation succeeded. Elapsed time (ms): 30 (TaskId:31)
    2019-08-13T14:51:05.2860086Z                      Read config file successfully (TaskId:31)
    2019-08-13T14:51:05.2860237Z                      Overwriting analysis settings for a test project... (TaskId:31)
    2019-08-13T14:51:05.2860426Z                      Output Property: SQRuleSetFilePath=d:\agents\agent02-DEV\_work\17\.sonarqube\conf\SonarQubeRoslyn-cs-test.ruleset (TaskId:31)
    2019-08-13T14:51:05.2860544Z                      Output Item(s): 
    2019-08-13T14:51:05.2860661Z                          SQAnalyzerFilePaths=
    2019-08-13T14:51:05.2860812Z                              C:\Users\tfstask\AppData\Local\Temp\.sonarqube\resources\0\Google.Protobuf.dll
    2019-08-13T14:51:05.2860981Z                              C:\Users\tfstask\AppData\Local\Temp\.sonarqube\resources\0\SonarAnalyzer.CSharp.dll
    2019-08-13T14:51:05.2861146Z                              C:\Users\tfstask\AppData\Local\Temp\.sonarqube\resources\0\SonarAnalyzer.dll (TaskId:31)
    2019-08-13T14:51:05.2861336Z                      Output Item(s): SQAdditionalFiles=d:\agents\agent02-DEV\_work\17\.sonarqube\conf\cs\SonarLint.xml (TaskId:31)
    2019-08-13T14:51:05.2861487Z                    Done executing task "GetAnalyzerSettings". (TaskId:31)

which may or may not be related.

While i was writing this post, i decided to run Test Assembly right after msbuild, and what do you know? Now all files are classified as source files.
I am mighty confused (probably because i’m illiterate ape), but if someone could provide me some insight in what’s going on - i would be grateful. :slight_smile:

After successful results on one branch, i decided to attempt to replicate my stunning success on another branch. And apparently, 1 of 3 commits i did to the 1st branch while fiddling with .csproj files was also a contributor:

    <!-- Mark the project as being a test project -->

Now, official docs never mention that you can do this (only true case), but this seemed to work in conjunction with running tests.
Now, I’m even more confused than before! :smiley:

Hi @Vlkmr - welcome to the community!

The scanner uses a number of different criteria to decide whether a project is a test project or not. Unfortunately, they are not all documented in one place - the docs refer to some of them (see “Detection of Test Projects”), and others are described in the targets file.

Here’s what I think is a comprehensive list:

  • the project file contains <SonarQubeTestProject>false</SonarQubeTestProject> or <SonarQubeTestProject>true</SonarQubeTestProject>
  • the project file contains the MSTest ProjectTypeGuid: 3AC096D0-A1C2-E12C-1390-A8335801FDAB
  • the project file contains the legacy Service GUID {82A7F48D-3B50-4B1E-B82E-3ADA8210C358} that added by the Test Explorer to mark a project as a containing tests
  • the project file contains the ProjectCapability TestContainer (for new SDK-style MSBuild projects)
  • the project file name matches the RegEx set in the property sonar.msbuild.testProjectPattern, which by default is .*Tests?\.(cs|vb)proj$
  • a couple of special cases where the build process creates temporary projects (e.g. Microsoft Fakes, WPF). Note that the “real” MSBuild projects are still analyzed as non-test projects; only the temp projects are not analysed.

I’ve created a ticket to update the documentation: S4MSB #774

Also, ticket #770 is about improving the logging during build so it should be easy to tell from the build output why a project is being classified as a test project.

thanks for the welcome and sorry for the late response. :slight_smile:
Updating docs will surely help. But i have a couple more questions if you could answer them:

Do these conditions have different priorities? I.e. does

    <!-- Mark the project as being a test project -->

override any other conditions to qualify a project as test one that SQ detected? From my experience, i’d say no, but it would nice if it did.

Got any insight as to why must test run before analysis? Without them, SQ considers all files as test files regardless if property group is specified or not.

Thanks in advance.

Setting ‘’ explicitly to true of false takes precedence i.e. the other conditions won’t be evaluated.

I’m not sure what you mean. Do you mean why do the tests have to be executed before the Azure Pipeline step Run Code Analysis? If so, that’s because Run Code Analysis step uploads the test results/coverage to SonarQube/Cloud.

Whether tests are run or not should not have any impact on how the files are classified. The classification is done as part of the MSBuild step, and the outcome written to .sonarqube\out\[numerical per-project folder]\ProjectInfo.xml. Running vstests won’t change what is in that file.

Update: I’ve added this page to the S4MSB wiki explaining the treatment and categorisation of test/product projects.

1 Like