Is there up-to-date definitive documentation for .NET Core code coverage/

davidkeaveny · May 22, 2023, 1:01am

I have the following setup:

ALM used - GitHub
CI system used - GitHub Actions

Scanner command used when applicable

.\.sonar\scanner\dotnet-sonarscanner begin /k:"${{ env.SONAR_PROJECT_ID }}" /o:"stp" /d:sonar.login="${{ secrets.SONAR_TOKEN }}" /d:sonar.host.url="https://sonarcloud.io" /v:${{ steps.gitversion.outputs.FullSemVer }} /d:sonar.cs.vscoveragexml.reportsPath="${{ env.TEST_OUTPUT }}/${{ env.TEST_FILE }}"
dotnet-coverage collect "dotnet test ${{ env.BUILD_SOLUTION }} --configuration ${{ env.BUILD_CONFIGURATION }} /p:Version=${{ steps.gitversion.outputs.FullSemVer }} --filter ""${{ env.TEST_FILTER }}""" -f xml -o "${{ env.TEST_OUTPUT }}/${{ env.TEST_FILE }}"
.\.sonar\scanner\dotnet-sonarscanner end /d:sonar.login="${{ secrets.SONAR_TOKEN }}"

Languages of the repository - C#
Error observed; the workflow completes successfully and publishes the results to SonarCloud; however SonarCloud reports that one of the files that is being modified by the PR has no code coverage; yet the code coverage was executed and I currently have 97% coverage when run locally

Here’s the relevant portion of the logs, where the test project is executed and coverage run:

Stp.Etl.Client.Tests -> D:\a\stp\stp\submissions\tests\Stp.Etl.Client.Tests\bin\Release\net6.0\Stp.Etl.Client.Tests.dll
  Sonar: (Stp.Etl.Client.Tests.csproj) Project processed successfully
Test run for D:\a\stp\stp\submissions\tests\Stp.Etl.Client.Tests\bin\Release\net6.0\Stp.Etl.Client.Tests.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:   341, Skipped:     0, Total:   341, Duration: 2 s - Stp.Etl.Client.Tests.dll (net6.0)
Code coverage results: TestResults/stp-etl-coverage.xml.

Looking at the output of dotnet-sonarscanner when evaluating the test project, I see:

INFO: ------------- Run sensors on module Stp.Etl.Client.Tests
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=0ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Roslyn version: 4.5.0.0
INFO: Language version: CSharp11
INFO: Concurrent execution: enabled
INFO: Sensor C# Analysis Log [csharp] (done) | time=0ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: Sensor TextAndSecretsSensor is restricted to changed files only
INFO: 8 source files to be analyzed
INFO: 8/8 source files have been analyzed
INFO: Sensor TextAndSecretsSensor [text] (done) | time=74ms
INFO: Sensor VB.NET Project Type Information [vbnet]
INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=0ms
INFO: Sensor VB.NET Analysis Log [vbnet]
INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=0ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=0ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=0ms
INFO: Sensor CSS Rules [javascript]
INFO: Sensor CSS Rules is restricted to changed files only
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=4ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: Sensor IaC Docker Sensor is restricted to changed files only
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=0ms
INFO: Sensor Serverless configuration file sensor [security]
INFO: 0 Serverless function entries were found in the project
INFO: 0 Serverless function handlers were kept as entrypoints
INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM template file sensor [security]
INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM Inline template file sensor [security]
INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms

and the same for the assembly which is being tested:

INFO: ------------- Run sensors on module Stp.Etl.Client
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=0ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Roslyn version: 4.5.0.0
INFO: Language version: CSharp11
INFO: Concurrent execution: enabled
INFO: Sensor C# Analysis Log [csharp] (done) | time=0ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor HTML [web]
INFO: Sensor HTML is restricted to changed files only
INFO: Sensor HTML [web] (done) | time=0ms
INFO: Sensor XML Sensor [xml]
INFO: Sensor XML Sensor is restricted to changed files only
INFO: Sensor XML Sensor [xml] (done) | time=79ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: Sensor TextAndSecretsSensor is restricted to changed files only
INFO: 3 source files to be analyzed
INFO: 3/3 source files have been analyzed
INFO: Sensor TextAndSecretsSensor [text] (done) | time=15ms
INFO: Sensor VB.NET Project Type Information [vbnet]
INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=0ms
INFO: Sensor VB.NET Analysis Log [vbnet]
INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=0ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=0ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=0ms
INFO: Sensor CSS Rules [javascript]
INFO: Sensor CSS Rules is restricted to changed files only
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=0ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: Sensor IaC Docker Sensor is restricted to changed files only
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=0ms
INFO: Sensor Serverless configuration file sensor [security]
INFO: 0 Serverless function entries were found in the project
INFO: 0 Serverless function handlers were kept as entrypoints
INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM template file sensor [security]
INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM Inline template file sensor [security]
INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms

Now my solution does have multiple test projects, but the coverage for them should be merged by the dotnet-coverage collect "dotnet test ..." command.

So to summarise - SonarCloud is reporting quality gate failures on my code coverage with a value of 0% affected lines covered, yet the file(s) that are affected have adequate code coverage when tested locally. There doesn’t appear to be anything in the logs to say that the coverage reports are not being generated or not being processed.

Colin · May 22, 2023, 4:12pm

Hello!

To start: it should be sonar.cs.vscoveragexml.reportsPaths instead of sonar.cs.vscoveragexml.reportsPath.

Can you first make that change, and see what additional information you get about test coverage import (if it’s not simply successful after this change)?

davidkeaveny · May 23, 2023, 12:37am

This site needs an “I’m an idiot” button; spelling property name correctly does indeed make the expected data flow through, and now I have all the coverage reports that I could ask for. Thanks for your help!

Colin · May 23, 2023, 7:55am

Don’t sweat it. Across languages, we had several iterations of reportsPath, reportPaths, reportsPaths (which also changed when we started accepting multiple reports)… it has been settled for some years now but I always have to look it up myself!

system · May 30, 2023, 7:55am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.