Can't get SonarCloud to pickup coverage reports

SonarCloud
, , , ,
1
  • ALM used: Azure DevOps
  • CI system used: Azure DevOps
  • Languages of the repository: C#

Dear community,

We are facing problems reporting code coverage to SonarCloud with a C# .NET 4.7.2 project. We are using Azure DevOps as CI infrastructure. Our abbreviated and redacted pipeline yml looks like this:

steps:
    - task: SonarCloudPrepare@2
      displayName: 'Prepare analysis on Sonar Cloud'
      inputs:
        SonarCloud: 'AzureDevOpsServiceConnectionName'
        organization: 'org'
        scannerMode: 'MSBuild'
        projectKey: 'projectKey'
        projectName: 'project'
        extraProperties: |
            sonar.verbose=true
            sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)\**\*.trx

    - task: MSBuild@1
      displayName: 'Building the backend solution'
      inputs:
        msbuildVersion: 'latest'
        solution: 'src/backend/Project.sln'
        msbuildArchitecture: 'x64'
        platform: 'x64'
        configuration: 'Debug'
        clean: true
        msbuildArguments: '/restore /m /nr:false /p:UseSharedCompilation=true'

    - task: VSTest@2
      displayName: 'Execute tests and measure code coverage'
      continueOnError: true
      inputs:
          distributionBatchType: 'basedOnTestCases'
          testSelector: 'testAssemblies'
          testAssemblyVer2: |
            **\Project.Common.Tests
            **\Project.Storage.Tests.dll
            !**\*TestAdapter.dll
            !**\obj\**
          searchFolder: '$(System.DefaultWorkingDirectory)/src/backend'
          vsTestVersion: 'latest'
          runInParallel: false
          codeCoverageEnabled: true
          rerunFailedTests: true
          rerunType: 'basedOnTestFailureCount'
          rerunFailedTestCasesMaxLimit: '10'
          rerunMaxAttempts: '1'

    - task: SonarCloudAnalyze@2
      displayName: 'Run Code Analysis'
      inputs:
        jdkversion: 'JAVA_HOME_17_X64'

    - task: SonarCloudPublish@2
      displayName: 'Publish Quality Gate result'
      inputs:
        pollingTimeoutSec: '300'

We have tried many different combinations of config parameter for the SonarCloud tasks, but never managed to see the coverage in the SonarCloud UI. It only shows the message “A few extra steps are needed for SonarCloud to analyze your code coverage.” The code coverage is visible in the Azure DevOps UI, so the measuring of the coverage works, only the sending to SonarCloud doesn’t work yet.

We dug through the logs and runner file system and found the following things.

Inside the Agent.TempDirectory there are the test results as a trx file (1.1M) and the code coverage as a binary .coverage file (2.7M).

TestResults/VssAdministrator_fv-az763-589_2024-08-15_12_08_19
TestResults/VssAdministrator_fv-az763-589_2024-08-15_12_08_19/In
TestResults/VssAdministrator_fv-az763-589_2024-08-15_12_08_19/In/fv-az763-589
TestResults/VssAdministrator_fv-az763-589_2024-08-15_12_08_19/In/fv-az763-589/VssAdministrator_fv-az763-589_2024-08-15.12_10_09.coverage
TestResults/VssAdministrator_fv-az763-589_2024-08-15_12_08_19.trx
TestResults/1088d870-e54f-4076-89e4-f126a8debc3f
TestResults/1088d870-e54f-4076-89e4-f126a8debc3f/VssAdministrator_fv-az763-589_2024-08-15.12_10_09.coverage

I’m not able to upload the complete log file publicly but can send to SonarCloud staff privately if helpful or required. But here is the redacted end of the log file of the SonarCloudAnalyze task:

...

2024-08-15T14:48:59.9540109Z ------------------------------------------------------------------------
2024-08-15T14:48:59.9540438Z 14:48:59.646  Writing processing summary to D:\a\1\.sonarqube\out\ProjectInfo.log
2024-08-15T14:48:59.9540788Z Calling the TFS Processor executable...
2024-08-15T14:48:59.9541279Z Executing file D:\a\_tasks\SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255\2.3.2\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.TFSProcessor.exe
2024-08-15T14:48:59.9541851Z   Args: ConvertCoverage D:\a\1\.sonarqube\conf\SonarQubeAnalysisConfig.xml D:\a\1\.sonarqube\out\sonar-project.properties 
2024-08-15T14:48:59.9542139Z   Working directory: D:\a\1
2024-08-15T14:48:59.9542341Z   Timeout (ms):-1
2024-08-15T14:48:59.9542535Z   Process id: 1064
2024-08-15T14:48:59.9542911Z Property 'sonar.cs.vstest.reportsPaths' provided, skipping the search for TRX files in default folders...
2024-08-15T14:49:00.1333093Z Did not find any binary coverage files in the expected location.
2024-08-15T14:49:00.1355303Z Falling back on locating coverage files in the agent temp directory.
2024-08-15T14:49:00.1360175Z Searching for coverage files in D:\a\_temp
2024-08-15T14:49:00.1379488Z All matching files: count=2
2024-08-15T14:49:00.1381341Z 	D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coverage
2024-08-15T14:49:00.1382942Z 	D:\a\_temp\TestResults\VssAdministrator_fv-az406-130_2024-08-15_14_42_20\In\fv-az406-130\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coverage
2024-08-15T14:49:00.6926236Z Unique coverage files: count=1
2024-08-15T14:49:00.6928644Z 	D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coverage
2024-08-15T14:49:00.7088147Z Converting coverage file 'D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coverage' to 'D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coveragexml'.
2024-08-15T14:49:01.0502547Z Coverage report conversion completed successfully.
2024-08-15T14:49:01.0595256Z Process returned exit code 0

...

2024-08-15T14:50:36.5477688Z 14:50:35.833 INFO: Sensor C# Tests Coverage Report Import [csharp]
2024-08-15T14:50:36.5478828Z 14:50:35.834 DEBUG: Analyzing coverage with wildcardPatternFileProvider with base dir 'D:\a\1\.' and file separator '\'.
2024-08-15T14:50:36.5480279Z 14:50:35.835 DEBUG: Pattern matcher extracted prefix/absolute path 'D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coveragexml' from the given pattern 'D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coveragexml'.
2024-08-15T14:50:36.5482827Z 14:50:35.835 DEBUG: Pattern matcher returns a single file: 'D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coveragexml'.
2024-08-15T14:50:36.5485021Z 14:50:35.835 DEBUG: The current user dir is 'D:\a\1'.
2024-08-15T14:50:36.5487176Z 14:50:35.835 INFO: Parsing the Visual Studio coverage XML report D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coveragexml
2024-08-15T14:50:36.5488760Z 14:50:35.972 DEBUG: The file 'C:\projects\moq4\src\Moq\obj\TypeNameFormatter.cs' does not have a deterministic build path and is either not indexed or does not have a supported language. Will skip this coverage entry. Verify sonar.sources in .sonarqube\out\sonar-project.properties.
2024-08-15T14:50:36.5491422Z 14:50:35.972 DEBUG: The file 'C:\projects\moq4\src\Moq\ActionObserver.cs' does not have a deterministic build path and is either not indexed or does not have a supported language. Will skip this coverage entry. Verify sonar.sources in .sonarqube\out\sonar-project.properties.
... more Moq files ...
2024-08-15T14:50:36.5691983Z 14:50:36.110 DEBUG: The file 'C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\AdapterSettings.cs' does not have a deterministic build path and is either not indexed or does not have a supported language. Will skip this coverage entry. Verify sonar.sources in .sonarqube\out\sonar-project.properties.
2024-08-15T14:50:36.5693097Z 14:50:36.110 DEBUG: The file 'C:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\CategoryList.cs' does not have a deterministic build path and is either not indexed or does not have a supported language. Will skip this coverage entry. Verify sonar.sources in .sonarqube\out\sonar-project.properties.
... more NUnit files ...
2024-08-15T14:50:36.5818366Z 14:50:36.278 INFO: Adding this code coverage report to the cache for later reuse: D:\a\_temp\TestResults\f06cb2d6-aa47-4195-a50c-31baef78d4b4\VssAdministrator_fv-az406-130_2024-08-15.14_44_29.coveragexml
2024-08-15T14:50:36.5819487Z 14:50:36.284 DEBUG: Analyzing coverage after aggregate found '476' coverage files.
2024-08-15T14:50:36.5975382Z 14:50:36.297 DEBUG: Skipping 'D:\a\1\s\src\backend\Redacted.Tests\Tests.Common\VirtualServiceFabric\ContextMockFactory.cs' as it is a test file.
2024-08-15T14:50:36.5976858Z 14:50:36.301 DEBUG: Skipping 'D:\a\1\s\src\backend\Redacted.Tests\Tests.Common\Utils.cs' as it is a test file.
... more of our test code files ...
2024-08-15T14:50:36.6238563Z 14:50:36.447 DEBUG: Skipping 'D:\a\1\s\src\backend\Redacted.Tests\Tests.Common\RandomValueGenerator.cs' as it is a test file.
2024-08-15T14:50:36.6240318Z 14:50:36.449 DEBUG: The total number of file count statistics is '476'.
2024-08-15T14:50:36.6243001Z 14:50:36.457 INFO: Coverage Report Statistics: 476 files, 402 main files, 402 main files with coverage, 74 test files, 0 project excluded files, 0 other language files.
2024-08-15T14:50:36.6244800Z 14:50:36.457 INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=624ms
2024-08-15T14:50:36.6245549Z 14:50:36.462 INFO: Sensor C# Unit Test Results Import [csharp]
2024-08-15T14:50:36.6246431Z 14:50:36.463 DEBUG: Pattern matcher extracted prefix/absolute path 'D:\a\_temp' from the given pattern 'D:\a\_temp\**\*.trx'.
2024-08-15T14:50:36.6247235Z 14:50:36.463 DEBUG: Gathering files for wildcardPattern '**\*.trx'.
2024-08-15T14:50:36.6247996Z 14:50:36.465 DEBUG: Pattern matcher returns '1' files.
2024-08-15T14:50:36.6249255Z 14:50:36.465 DEBUG: The current user dir is 'D:\a\1'.
2024-08-15T14:50:36.6250167Z 14:50:36.466 INFO: Parsing the Visual Studio Test Results file 'D:\a\_temp\TestResults\VssAdministrator_fv-az406-130_2024-08-15_14_42_20.trx'.
2024-08-15T14:50:36.6250978Z 14:50:36.468 DEBUG: Parsed Visual Studio Test Times - duration: 145524.
2024-08-15T14:50:36.6251873Z 14:50:36.543 DEBUG: Parsed Visual Studio Test Counters - total: 531, failed: 0, errors: 0, timeout: 0, aborted: 0, executed: 528.
2024-08-15T14:50:36.6252601Z 14:50:36.566 INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=104ms
2024-08-15T14:50:36.6253175Z 14:50:36.566 INFO: Sensor Zero Coverage Sensor
2024-08-15T14:50:36.9423300Z 14:50:36.907 INFO: Sensor Zero Coverage Sensor (done) | time=341ms
2024-08-15T14:50:36.9426546Z 14:50:36.917 INFO: SCM Publisher SCM provider for this project is: git
2024-08-15T14:50:36.9427903Z 14:50:36.919 INFO: SCM Publisher 2 source files to be analyzed
2024-08-15T14:50:36.9432576Z 14:50:36.920 DEBUG: loading config FileBasedConfig[C:\Users\VssAdministrator\.config\jgit\config]
2024-08-15T14:50:36.9433549Z 14:50:36.923 DEBUG: Collecting committed files
2024-08-15T14:50:37.0765089Z 14:50:37.073 DEBUG: Collecting committed files (done) | time=151ms
2024-08-15T14:50:37.0792799Z 14:50:37.075 DEBUG: Using GIT_NATIVE_BLAME strategy to blame files
2024-08-15T14:50:37.0820712Z 14:50:37.075 DEBUG: Looking for git command in the PATH using where.exe (Windows)
2024-08-15T14:50:37.2133148Z 14:50:37.212 DEBUG: Found git.exe at C:\agents\3.242.1\externals\git\cmd\git.exe
2024-08-15T14:50:37.2762458Z 14:50:37.275 DEBUG: Found GIT version: 2.45.2
2024-08-15T14:50:37.2780845Z 14:50:37.277 DEBUG: Blame file (native) src/backend/Redacted/EnergyHistoryRepository.cs
2024-08-15T14:50:37.2788144Z 14:50:37.277 DEBUG: Blame file (native) src/backend/Redacted/Backend/Services/ProductService.cs
2024-08-15T14:50:38.2227459Z 14:50:38.095 INFO: SCM Publisher 2/2 source files have been analyzed (done) | time=1176ms
2024-08-15T14:50:38.2233104Z 14:50:38.220 INFO: CPD Executor 366 files had no CPD blocks
2024-08-15T14:50:38.2241481Z 14:50:38.220 INFO: CPD Executor Calculating CPD for 1206 files
2024-08-15T14:50:38.2255560Z 14:50:38.221 DEBUG: Detection of duplications for D:/a/1/s/src/backend/Redacted/StandbyHistoryEntry.cs
2024-08-15T14:50:38.2929734Z 14:50:38.236 DEBUG: Detection of duplications for D:/a/1/s/src/backend/Redacted/Context.cs
... more production code files ...
2024-08-15T14:50:39.5421822Z 14:50:38.752 DEBUG: Detection of duplications for D:/a/1/s/src/backend/Redacted/CircuitBreakerDecorator.cs
2024-08-15T14:50:39.5422202Z 14:50:38.752 INFO: CPD Executor CPD calculation finished (done) | time=532ms
2024-08-15T14:50:39.5422494Z 14:50:39.278 INFO: SCM writing changed lines
2024-08-15T14:50:39.5422795Z 14:50:39.308 DEBUG: Merge base sha1: 2cd24f06637b79c1b47dc4b12637243b01ed841d
2024-08-15T14:50:39.5423295Z 14:50:39.457 DEBUG: SCM reported changed lines for 2 files in the branch
2024-08-15T14:50:39.5424398Z 14:50:39.458 INFO: SCM writing changed lines (done) | time=180ms
2024-08-15T14:50:44.4638655Z 14:50:44.462 DEBUG: MSBuild version found: 17.10.4.21802
2024-08-15T14:50:45.0049856Z 14:50:45.003 INFO: Analysis report generated in 6065ms, dir size=2 MB
2024-08-15T14:50:47.2306381Z 14:50:47.229 INFO: Analysis report compressed in 2225ms, zip size=1 MB
2024-08-15T14:50:47.2309029Z 14:50:47.229 INFO: Analysis report generated in D:\a\1\.sonarqube\out\.sonar\scanner-report
2024-08-15T14:50:47.2310894Z 14:50:47.229 DEBUG: Upload report
2024-08-15T14:50:47.2404421Z 14:50:47.235 DEBUG: --> POST https://sonarcloud.io/api/ce/submit?organization=orga&projectKey=redacted&projectName=redacted&characteristic=branch%3Dfeature%2Fmybranch&characteristic=branchType%3DSHORT (1796955-byte body)
2024-08-15T14:50:50.6364819Z 14:50:50.634 DEBUG: <-- 200 https://sonarcloud.io/api/ce/submit?organization=orga&projectKey=redacted&projectName=redacted&characteristic=branch%3Dfeature%2Fmybranch&characteristic=branchType%3DSHORT (3399ms, 44-byte body)
2024-08-15T14:50:50.6399535Z 14:50:50.639 INFO: Analysis report uploaded in 3410ms
2024-08-15T14:50:50.6424627Z 14:50:50.641 DEBUG: Report metadata written to D:\a\_temp\sonar\752607\cf489c66-e2cf-4cce-70cc-43306738d258\report-task.txt
2024-08-15T14:50:50.6426253Z 14:50:50.641 INFO: ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=redacted&branch=feature%2Fmybranch&resolved=false
2024-08-15T14:50:50.6427442Z 14:50:50.641 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
2024-08-15T14:50:50.6428550Z 14:50:50.641 INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=redacted
2024-08-15T14:50:50.7356585Z 14:50:50.734 DEBUG: --> GET https://api.sonarcloud.io/analysis/sensor-cache/prepare-write?organization=orga&project=redacted&branch=feature/mybranch
2024-08-15T14:50:51.1682684Z 14:50:51.166 DEBUG: <-- 200 https://api.sonarcloud.io/analysis/sensor-cache/prepare-write?organization=orga&project=redacted&branch=feature/mybranch (432ms, 1518-byte body)
2024-08-15T14:50:51.1711512Z 14:50:51.169 DEBUG: --> PUT https://analysis-sensorcache-eu-central-1-prod.s3.amazonaws.com/AY_redacted (569468-byte body)
2024-08-15T14:50:52.9106271Z 14:50:51.627 DEBUG: <-- 200 OK https://analysis-sensorcache-eu-central-1-prod.s3.amazonaws.com/AY_redacted (457ms, 0-byte body)
2024-08-15T14:50:53.9188192Z 14:50:51.627 INFO: Sensor cache published successfully
2024-08-15T14:50:53.9190483Z 14:50:51.630 DEBUG: Post-jobs : 
2024-08-15T14:50:53.9191224Z 14:50:51.633 INFO: Analysis total time: 1:23.556 s
2024-08-15T14:50:53.9192125Z 14:50:51.636 INFO: ------------------------------------------------------------------------
2024-08-15T14:50:53.9192710Z 14:50:51.636 INFO: EXECUTION SUCCESS
2024-08-15T14:50:53.9193412Z 14:50:51.636 INFO: ------------------------------------------------------------------------
2024-08-15T14:50:53.9194127Z 14:50:51.636 INFO: Total time: 1:48.516s
2024-08-15T14:50:53.9194596Z 14:50:51.785 INFO: Final Memory: 75M/254M
2024-08-15T14:50:53.9195239Z 14:50:51.786 INFO: ------------------------------------------------------------------------
2024-08-15T14:50:53.9195860Z 14:50:51.818 DEBUG: Cleanup org.eclipse.jgit.util.FS$FileStoreAttributes$$Lambda$290/0x000001c3332f9f68@2caa5d7c during JVM shutdown
2024-08-15T14:50:53.9196410Z Process returned exit code 0
2024-08-15T14:50:53.9196855Z The SonarScanner CLI has finished
2024-08-15T14:50:53.9197569Z 14:50:52.228  Post-processing succeeded.
2024-08-15T14:50:53.9209947Z ##[section]Finishing: Run Code Analysis

As far as we can tell, everything in log seems fine. The .coverage file is found and successfully converted to the .coveragexml format. Still, we don’t see any coverage info in on this page SonarCloud

We’ve read through all relevant documentation:

  • https_//community.sonarsource.com/t/coverage-troubleshooting-guide-for-net-code-coverage-import/37151
  • https_//docs.sonarsource.com/sonarqube/latest/analyzing-source-code/test-coverage/dotnet-test-coverage/
  • https_//docs.sonarsource.com/sonarqube/latest/analyzing-source-code/test-coverage/test-coverage-parameters/
  • https_//docs.sonarsource.com/sonarcloud/advanced-setup/ci-based-analysis/azure-pipelines/
  • https_//docs.sonarsource.com/sonarqube/latest/devops-platform-integration/azure-devops-integration/
  • https_//docs.sonarsource.com/sonarqube/latest/analyzing-source-code/scanners/sonarscanner-for-dotnet/
  • https_//docs.sonarsource.com/sonarqube/latest/analyzing-source-code/analysis-parameters/

We would be ever grateful for hints on what we are missing.

Kind regards

1 Like
3

Hey there.

I think you’re facing a bug that was already reported:

It only affects PRs/short-lived branches with no lines of code that can be covered by tests (0 new lines)

A fix is scheduled for an upcoming hardening sprint. You should be able to verify that coverage is read correctly by running an analysis (with coverage generated and passed to the scanner) on your main branch.

1 Like
4

We managed to solve this problem. We did add new fake code to trick the scanner into thinking there is an actual change in this branch, but I didn’t mention it in the post.

Because of the huge number of tests, we also decreased which tests are actually run on the branch, but we decreased it too much. There were no tests left that cover the new fake code.

After enabling all tests again, we saw a coverage percentage in the analysis report even on a short lived branch.

Your answer steered us in the correct direction. Thanks :pray:

5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.