Coverlet - Missing some coverage measures in Sonarqube Server compared to coverlet standalone report

[danzone (Daniele Gagliardi)]

Hi all, I’m configuring a CI pipeline on Jenkins for a DotNET project and I want to add a step of source code analysis with code ceoverage info. I performed several attempts both in the Jenkins CI server (on a Linux machine) and in my laptop (equipped with Win 11).
The project layout is the following:
root
|__project.sln
|__Module A
| |_____moduleA.csproj
|__Module B (dependant from Module A)
| |_____moduleB.csproj
|__Module C (not relevant for the analysis)
|_____moduleC.csproj
|__Test Module
|_____testmodule.csproj

When I execute the unit tests and compute coverage with coverlet, I obtain a report with measures both on Module A and Module B. Then the XML file is sent to SonarQube Server and in the dashboard I see code coverage mneasures just on Module A.
To have a more clear vision on coverage I generated the coverage report with report generator and I confirm that in this report I can have coverage info both on Module A and Module B.
Here relevant informations:

• SonarQube Server/Community Build 9.9.7
• Deployment: zip
• Database: PostgreSQL
• what am I trying to achieve: code coverage analysis consistent with what found by coverlet

Following the step I executed:

dotnet-sonarscanner.exe begin /k:"<project-key>" /d:sonar.token="<sonar-token>" /d:sonar.host.url="<sonar-server-url>" /d:sonar.cs.opencover.reportsPaths=TestResults/**/coverage.opencover.xml /d:sonar.cs.vstest.reportsPaths=TestResults/testResults.trx

dotnet build --no-incremental

dotnet test Test.Module/testmodule.csproj --logger:"trx;logfilename=testResults.trx" --results-directory TestResults --collect:"XPlat Code Coverage;Format=opencover"

 dotnet-sonarscanner.exe end /d:sonar.login="<sonar-token>"

Coverlet report generated with:

reportgenerator.exe -reports:.\TestResults\<path to>\coverage.opencover.xml -targetDir:coverlet/reports -reporttypes:HtmlInline_AzurePipelines

Here the relevant parts of the log:

PS C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project> dotnet-sonarscanner.exe begin /k:"project-key" /d:sonar.token="<sonar_token>" /d:sonar.host.url="http://localhost:9000"
SonarScanner for MSBuild 9.0.2
Using the .NET Core version of the Scanner for MSBuild
Pre-processing started.
Preparing working directories...
18:08:44.157  Updating build integration targets...
18:08:44.607  Using SonarQube v9.9.7.96285.
18:08:44.686  Fetching analysis configuration settings...
18:08:45.476  Provisioning analyzer assemblies for cs...
18:08:45.477  Installing required Roslyn analyzers...
18:08:45.478  Processing plugin: csharp version 8.51.0.59060
18:08:45.543  Processing plugin: vbnet version 8.51.0.59060
18:08:45.787  Provisioning analyzer assemblies for vbnet...
18:08:45.787  Installing required Roslyn analyzers...
18:08:45.787  Processing plugin: csharp version 8.51.0.59060
18:08:45.79  Processing plugin: vbnet version 8.51.0.59060
18:08:45.841  Incremental PR analysis: Base branch parameter was not provided.
18:08:45.841  Cache data is empty. A full analysis will be performed.
18:08:45.862  WARNING: Multi-Language analysis is enabled. If this was not intended and you have issues such as hitting your LOC limit or analyzing unwanted files, please set "/d:sonar.scanner.scanAll=false" in the begin step.
18:08:45.895  Pre-processing succeeded.
PS C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project> dotnet build --no-incremental
Ripristino completato (1,5)
  \ModuleA\ operazione riuscita (5,9s) → \ModuleA\bin\Development\net8.0\\ModuleA\.dll
  ModuleC\API operazione riuscita (4,5s) → ModuleC\ModuleC\API\bin\Debug\net8.0\ModuleC\API.dll
  \ModuleB\ operazione riuscita (8,6s) → \ModuleB\bin\Development\net8.0\\ModuleB\.dll
  Test.Module completato con 2 avvisi (6,7s) → Test.Module\bin\Debug\net8.0\Test.Module.dll
    C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\Test.Module\Istanza\Utils\EmailUtilitiesTest.cs(47,27): warning S2699: Add at least one assertion to this test case. (https://rules.sonarsource.com/csharp/RSPEC-2699)
    C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\Test.Module\Istanza\Utils\EmailUtilitiesTest.cs(26,27): warning S2699: Add at least one assertion to this test case. (https://rules.sonarsource.com/csharp/RSPEC-2699)

Compilazione completato con 2 avvisi in 22,1s
PS C:\Users\<my_home>\Lavori\Test\Analisi\MLPS\mlps-verificheperiodiche-api> dotnet build --no-incremental
Ripristino completato (1,5)
  \ModuleA\ operazione riuscita (5,9s) → \ModuleA\bin\Development\net8.0\\ModuleA\.dll
  ModuleC\API operazione riuscita (4,5s) → ModuleC\ModuleC\API\bin\Debug\net8.0\ModuleC\API.dll
  \ModuleB\ operazione riuscita (8,6s) → \ModuleB\bin\Development\net8.0\\ModuleB\.dll
  Test.Module completato con 2 avvisi (6,7s) → Test.Module\bin\Debug\net8.0\Test.Module.dll
    C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\Test.Module\Istanza\Utils\EmailUtilitiesTest.cs(47,27): warning S2699: Add at least one assertion to this test case. (https://rules.sonarsource.com/csharp/RSPEC-2699)
    C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\Test.Module\Istanza\Utils\EmailUtilitiesTest.cs(26,27): warning S2699: Add at least one assertion to this test case. (https://rules.sonarsource.com/csharp/RSPEC-2699)

Compilazione completato con 2 avvisi in 22,1s
PS C:\Users\<my_home>\Lavori\Test\Analisi\MLPS\mlps-verificheperiodiche-api> dotnet test Test.Module/Test.Module.csproj --logger:"trx;logfilename=testResults.trx" --results-directory TestResults --collect:"XPlat Code Coverage;Format=opencover"
Ripristino completato (0,9)
  \ModuleA\ operazione riuscita (2,2s) → \ModuleA\bin\Debug\net8.0\\ModuleA\.dll
  \ModuleB\ operazione riuscita (4,5s) → \ModuleB\bin\Debug\net8.0\\ModuleB\.dll
  Test.Module completato con 2 avvisi (3,8s) → Test.Module\bin\Debug\net8.0\Test.Module.dll
    C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\Test.Module\Istanza\Utils\EmailUtilitiesTest.cs(26,27): warning S2699: Add at least one assertion to this test case. (https://rules.sonarsource.com/csharp/RSPEC-2699)
    C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\Test.Module\Istanza\Utils\EmailUtilitiesTest.cs(47,27): warning S2699: Add at least one assertion to this test case. (https://rules.sonarsource.com/csharp/RSPEC-2699)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 8.0.11)
[xUnit.net 00:00:00.58]   Discovering: Test.Module
[xUnit.net 00:00:00.79]   Discovered:  Test.Module
[xUnit.net 00:00:00.79]   Starting:    Test.Module
[xUnit.net 00:00:42.78]   Finished:    Test.Module
AVVISO: il file dei risultati C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\TestResults\testResults.trx verrà sovrascritto
File di risultati: C:\Users\<my_home>\Lavori\Test\Analisi\projects\my-project\TestResults\testResults.trx
  Test.Module                                                              GenerateCoverageResult (71,6s)

+-------------------------+--------+--------+--------+
| Module                  | Line   | Branch | Method |
+-------------------------+--------+--------+--------+
| \ModuleB\ | 80.42% | 75.36% | 83.01% |
+-------------------------+--------+--------+--------+
| \ModuleA\ | 99.31% | 100%   | 99.31% |
+-------------------------+--------+--------+--------+

+---------+--------+--------+--------+
|         | Line   | Branch | Method |
+---------+--------+--------+--------+
| Total   | 81.16% | 75.36% | 85.88% |
+---------+--------+--------+--------+
| Average | 89.86% | 87.68% | 91.16% |
+---------+--------+--------+--------+
  Test.Module test operazione riuscita 71,6

Riepilogo test: totale: 278; non riuscito: 0; riuscito: 278; ignorato: 0; durata: 56,8s
Compilazione completato con 2 avvisi in 84,6s

After the sonar scanner ends (dotnet-sonarscanner.exe end /d:sonar.login="<sonar_token>"), In SonarQube only ModuleA is considered, and nothing is measured on Module B (in section code of the dashboard no number of lines of code are displayed and coverage isn’t reported)

What am I doing wrong?

Thanks in advance for your help

[Colin]

Hey there.

Based on what your describing, ModuleB is probably being considered a test project. What does that mean? Read about it here.

If that’s not the case, I’d like to see the logs from the scanner about importing coverage.

How to find logs about importing code coverage

When coverage data isn’t appearing as expected in SonarQube Server / Community Build or SonarQube Cloud, the first place you should look is the scanner logs. Where do I find the scanner logs? This will likely be broken out into its own guide soon The Scanner logs are the output of executing one of the scanners, either directly or wrapped by one of many integrations, such as the SonarScanner for Jenkins, SonarScanner for Azure DevOps, SonarQube/SonarCloud GitHub Actions or SonarQube /…

[danzone (Daniele Gagliardi)]

Dear Colin, thank you! That was the reason, Module B considered as a test project. I added following snippet to moduleB.csproj file:

<PropertyGroup>
    <SonarQubeTestProject>false</SonarQubeTestProject>
</PropertyGroup>

and Sonar scanner analyzed also Module B (with code coverage).
Cheers,

Daniele

PS: I edited my previous post in order to set the right parameters I actually used in the sonar scanner begin, to make this post useful for others (I previously forgot to add parameters related to the position of test results and coverage results)