Unable to setup SonarQube dotnet net6.0 xunit test results

[webartoli (Claudio Bartoli)]

I’m trying to setup sonarqube on dotnet core net6.0 project.

The main goal is to have tests and coverage setted up, to achieve this goal I followed this community post as reference - SonarQube Could not import unit test report for xUnit

So I setted up:

• XunitXml.TestLogger as Test result reporter (xunit format)
• coverlet.collector as Coverage Reporter (opencover format)

I setted up a simple project to make katasGym.Tests.csproj file looks like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="XunitXml.TestLogger" Version="3.0.70" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.1.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\Gym\Gym.csproj" />
  </ItemGroup>

</Project>

I’m running scripts in the following order

• build
• tests
• sonarqube

Build

$ dotnet build

Tests

$ dotnet test \
  --test-adapter-path:. \
  --logger:"xunit;LogFilePath=../artifacts/xunit/{assembly}-test-result.xml" \
  --collect:"XPlat Code Coverage" \
  -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover

SonarQube

$ dotnet sonarscanner begin /d:sonar.login="$SONAR_TOKEN" \
    /k:"$SONAR_PROJECT_KEY" \
    /d:sonar.host.url="$SONAR_HOST_URL" \
    /d:sonar.cs.xunit.reportsPaths="artifacts/xunit/*.xml" \
    /d:sonar.cs.opencover.reportsPaths="**/TestResults/**/coverage.opencover.xml"
$ dotnet build --no-incremental
$ dotnet sonarscanner end /d:sonar.login="$SONAR_TOKEN"

Results

Coverage is correctly detected

Tests are not

The content of the the artifacts\xunit\Gym.Tests-test-result.xml file is:

<assemblies timestamp="10/21/2022 08:43:17">
  <assembly name="C:\git\lab\Gym\Gym.Tests\bin\Debug\net6.0\Gym.Tests.dll" run-date="2022-10-21" run-time="08:43:17" total="9" passed="9" failed="0" skipped="0" time="2.665" errors="0">
    <errors />
    <collection total="5" passed="5" failed="0" skipped="0" name="Test collection for Gym.Tests" time="0.006">
      <test name="Gym.Tests.Base10Calculator.Sum" type="Gym.Tests" method="Base10Calculator.Sum" time="0.0052714" result="Pass">
        <traits />
      </test>
      <test name="Gym.Tests.Base10Calculator.DivisionBy0" type="Gym.Tests" method="Base10Calculator.DivisionBy0" time="0.0008151" result="Pass">
        <traits />
      </test>
      <test name="Gym.Tests.Base10Calculator.Subtract" type="Gym.Tests" method="Base10Calculator.Subtract" time="0.0000837" result="Pass">
        <traits />
      </test>
      <test name="Gym.Tests.Base10Calculator.Divide" type="Gym.Tests" method="Base10Calculator.Divide" time="0.0000721" result="Pass">
        <traits />
      </test>
      <test name="Gym.Tests.Base10Calculator.Multiply" type="Gym.Tests" method="Base10Calculator.Multiply" time="0.0001420" result="Pass">
        <traits />
      </test>
    </collection>
    <collection total="1" passed="1" failed="0" skipped="0" name="Test collection for Gym.Tests.Base2Calculator" time="0.005">
      <test name="Gym.Tests.Base2Calculator.Sum" type="Gym.Tests.Base2Calculator" method="Sum" time="0.0053802" result="Pass">
        <traits />
      </test>
    </collection>
    <collection total="3" passed="3" failed="0" skipped="0" name="Test collection for Gym.Tests.Base8Calculator" time="0.006">
      <test name="Gym.Tests.Base8Calculator.Sum" type="Gym.Tests.Base8Calculator" method="Sum" time="0.0053788" result="Pass">
        <traits />
      </test>
      <test name="Gym.Tests.Base8Calculator.Subtract" type="Gym.Tests.Base8Calculator" method="Subtract" time="0.0002531" result="Pass">
        <traits />
      </test>
      <test name="Gym.Tests.Base8Calculator.Division" type="Gym.Tests.Base8Calculator" method="Division" time="0.0005329" result="Pass">
        <traits />
      </test>
    </collection>
  </assembly>
</assemblies>

What am I missing on test results setup ?

[Colin]

Hey there.

The details of Tests aren’t available for C# – only the raw count. So you have everything configured as well as it can be. Take a look at this post here:

MSBuild Scanner missing unit tests details for NUnit 3

Hi @hobovsky For the sonar-dotnet plugins (C# and VB .NET), we don’t push unit test results data to SonarQube. Historically, it has been a missing feature of the plugin and we’ve had issue #886 to track it. However, recently we decided to close the issue as Won't fix - you can read the motivation in this comment. Having said that, you can import test data using Generic Test Data and it will work (see DotNet core code coverage - #7 by Thibault_DESAULES).

[webartoli (Claudio Bartoli)]

@Colin

The details of Tests aren’t available for C#

I didn’t expect this, good to know and thanks for your response.

Can you confirm that you don’t expect the number of tests at this point either?

It’s something that’s often convenient that we’re using in other languages.

[Colin]

Correct. There are no file-level measures on tests when analyzing C#.