CodeCoverage no longer reported in SonarQube

[milbrandt (Volker Milbrandt)]

Must-share information (formatted with Markdown):

• which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension) 10.4
• how is SonarQube deployed: zip
• what are you trying to achieve: Reporting of code coverage to SonarQube
• what have you tried so far to achieve this

What was changed / updated:

• .NET SDK 8.0.200 to 8.0.300
• Visual Studio 2022 to 17.10.0
• Microsoft.NET.Test.Sdk from 17.9.0 to 17.10.0
• MSTest from 3.3.1 to 3.4.0
• SonarAnalyzer.CSharp 9.23.2.88755 to 9.25.1.91650

Build pipeline as such (azure-pipelines.yml) remaind unchanged.

In Azure Pipelines the code coverage is still displayed.

Do not share screenshots of logs – share the text itself (bonus points for being well-formatted)!

  Process id: 4100
Microsoft (R) Coverage Collection Tool Version 17.0.24162.516205


Copyright (c) Microsoft Corporation.  All rights reserved.




CodeCoverage.exe is deprecated. Use dotnet-coverage global tool or Microsoft.CodeCoverage.Console instead. Some features of CodeCoverage.exe will be removed in next release of Visual Studio.



Usage: CodeCoverage <command>





Process returned exit code -1
Coverage report conversion completed successfully.
##[error]Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).
Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).
##[error]Check that the downloaded code coverage file (D:\_1\_temp\95187fae-3522-4760-a3c4-b83266de67a4\svc-buildtfs_SRVCS0000013_2024-05-27.13_22_51.coverage) is valid by opening it in Visual Studio. If it is not, check that the internet security settings on the build machine allow files to be downloaded from the Team Foundation Server machine.
Check that the downloaded code coverage file (D:\_1\_temp\95187fae-3522-4760-a3c4-b83266de67a4\svc-buildtfs_SRVCS0000013_2024-05-27.13_22_51.coverage) is valid by opening it in Visual Studio. If it is not, check that the internet security settings on the build machine allow files to be downloaded from the Team Foundation Server machine.
The TFS Processor has finished
Calling the SonarScanner CLI...

[Colin]

Hey there.

@milbrandt

• Microsoft.NET.Test.Sdk from 17.9.0 to 17.10.0

What happens if you revert this? There happens to be a Github issue (posted 12 minutes ago) claiming that that code coverage is broken with 17.10.0.

Code coverage seems to be broken in version 17.10.0

[norbert-groen]

Today I filed an issue against the Visual Studio test platform.

Code coverage seems to be broken in version 17.10.0 · Issue #5072 · microsoft/vstest (github.com)

It seems Codecoverage.exe from vstest is broken, but then again it is deprecated.
So is there a planning to replace this tool?
The tool itself states it should be replaced for: Microsoft.CodeCoverage.Console.

For now the problem is solved with using an “older” version of vstest.
But in the future that will become a problem.

Norbert.

[Colin]

Ah, hey @norbert-groen. I was really hoping you weren’t using SonarQube and these were independent issues.

@Colin

There happens to be a Github issue (posted 12 minutes ago) claiming that that code coverage is broken with 17.10.0.

I’m going to flag this for attention by our .NET experts.

[norbert-groen]

Thanks!

[milbrandt (Volker Milbrandt)]

I know of the CodeCoverage.exe is deprecated. We “fixed” it on the current repo by migration to coverlet and adjusting the pipeline.

[MJHDiepman (Mathieu Diepman)]

we had the same symptom and in our case it was caused by the fact that the projects containing logic were classified as Test projects due to a reference to Moq and excluded from coverage:

INFO: Found 19 MSBuild C# projects: 3 MAIN projects. 16 TEST projects.

This can be solved in two ways:

1. Remove reference to Moq from projects containing logic
2. Explicitly tell Sonar what kind of project it is via the .csproj:
   <SonarQubeTestProject>true/false</SonarQubeTestProject>

[Roland75 (Roland de Herder)]

Adding

<SonarQubeTestProject>false</SonarQubeTestProject>

to my csproj-files results in VS not loading the projects since this is an unrecognized tag.

[milbrandt (Volker Milbrandt)]

Where did you put it? In an Itemgroup or PropetyGroup? The following extract works for the property SonarQubeTestProject

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net8.0</TargetFramework>
        <SonarQubeTestProject>true</SonarQubeTestProject>
    </PropertyGroup>

[Roland75 (Roland de Herder)]

In an ItemGroup. Moved it to a PropertyGroup and now it works. Thanks.

[anon35261157]

Any news here?

We got similar issue:

Executing file C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
  Args: analyze ****
  Timeout (ms):60000
  Process id: 1488
CodeCoverage.exe is deprecated. Use dotnet-coverage global tool or Microsoft.CodeCoverage.Console instead. Some features of CodeCoverage.exe will be removed in next release of Visual Studio.


Process returned exit code -1
##[error]Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).
Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).
##[error]Check that the downloaded code coverage file ... is valid by opening it in Visual Studio. If it is not, check that the internet security settings on the build machine allow files to be downloaded from the Team Foundation Server machine.
Check that the downloaded code coverage file ***.cobertura.xml) is valid by opening it in Visual Studio. If it is not, check that the internet security settings on the build machine allow files to be downloaded from the Team Foundation Server machine.
Coverage report conversion completed successfully.
The TFS Processor has finished
Calling the SonarScanner CLI...

[Colin]

Hey all!

So, yesterday Sonar released new versions of the Azure DevOps Extension for SonarQube and for SonarCloud which can finally use newer versions of the SonarScanner for MSBuild, where we have migrated away from using CodeCoverage.exe.

Can you try upgrading to v6 of the Azure DevOps tasks for SonarQube or v2 for SonarCloud

[almolo (John)]

Hello, I’m having the same error. It’s coming from a very simple SonarCloudAnalyze task in Azure DevOps pipeline:

  - task: SonarCloudAnalyze@1
    displayName: Sonar Analysis

I’m assuming that behind the scenes this uses the latest available version (17.10.0)
But since I’m not using VisualStudioTestPlatformInstaller task or anything like that, I don’t know how to rollback from 17.10.0 to 17.9.0… Any suggestions?

[Colin]

Hey @almolo

Can you try configuring v2 of the tasks (SonarCloudAnalyze@2) and see if it all comes together? See my post above.

[almolo (John)]

Hi Colin, I just tried v2 for SonarCloud. Output is different and there are less errors in it, but ultimately it still fails to upload code coverage to SonarCloud.

Output from SonarCloudAnalyze@1 was like this:

Executing file C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
  Args: analyze /output:C:\a\3\s\TestResults\AzDevOps_aisar90cd000BJK_2024-06-05_13_02_40\In\aisar90cd000BJK\AzDevOps_aisar90cd000BJK_2024-06-05.13_02_41.cobertura.coveragexml C:\a\3\s\TestResults\AzDevOps_aisar90cd000BJK_2024-06-05_13_02_40\In\aisar90cd000BJK\AzDevOps_aisar90cd000BJK_2024-06-05.13_02_41.cobertura.xml 
  Working directory: C:\a\3\s\TestResults\AzDevOps_aisar90cd000BJK_2024-06-05_13_02_40\In\aisar90cd000BJK
  Timeout (ms):60000
  Process id: 5500
CodeCoverage.exe is deprecated. Use dotnet-coverage global tool or Microsoft.CodeCoverage.Console instead. Some features of CodeCoverage.exe will be removed in next release of Visual Studio.

Process returned exit code -1
Coverage report conversion completed successfully.
##[error]Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).
Check that the downloaded code coverage file (C:\a\3\s\TestResults\AzDevOps_aisar90cd000BJK_2024-06-05_13_02_40\In\aisar90cd000BJK\AzDevOps_aisar90cd000BJK_2024-06-05.13_02_41.cobertura.xml) is valid by opening it in Visual Studio. If it is not, check that the internet security settings on the build machine allow files to be downloaded from the Team Foundation Server machine.
Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).

Now output from SonarCloudAnalyze@2 is like this:

Converting coverage file 'C:\a\14\s\TestResults\AzDevOps_aisar90cd000BJ6_2024-06-05_13_35_33\In\aisar90cd000BJ6\AzDevOps_aisar90cd000BJ6_2024-06-05.13_35_33.cobertura.xml' to 'C:\a\14\s\TestResults\AzDevOps_aisar90cd000BJ6_2024-06-05_13_35_33\In\aisar90cd000BJ6\AzDevOps_aisar90cd000BJ6_2024-06-05.13_35_33.cobertura.coveragexml'.
Coverage report conversion completed successfully.
##[error]Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).
Check that the downloaded code coverage file (C:\a\14\s\TestResults\AzDevOps_aisar90cd000BJ6_2024-06-05_13_35_33\In\aisar90cd000BJ6\AzDevOps_aisar90cd000BJ6_2024-06-05.13_35_33.cobertura.xml) is valid by opening it in Visual Studio. If it is not, check that the internet security settings on the build machine allow files to be downloaded from the Team Foundation Server machine.
Failed to convert the binary code coverage reports to XML. No code coverage information will be uploaded to the server (SonarQube/SonarCloud).

As you can see it no longer complains about CodeCoverage.exe being deprecated and there is no process exit code -1. There’s even a line saying “Coverage report conversion completed successfully.” But then it fails again.

[Amit_T]

it works for me…

[almolo (John)]

A small update on my issues with SonarCloudAnalyze@2 - after fiddling around I realized v2 actually works, but only if my test coverage is in VS .coverage format. It does not work if coverage is in Cobertura format. Is cobertura not supported by SonarCloud v2?

[dellch (Chris)]

Edit: apologies, we’re not using SonarCloud, we have a self-install - so I’m not sure if it is relevant.

I was able to fix this issue by using version @6 of the tasks:

• - task: SonarQubePrepare@6
• - task: SonarQubeAnalyze@6
• - task: SonarQubePublish@6

Note that these versions are documented in SonarQube’s docs here, but not in Microsoft’s Documentation (link)

[Colin]

Thanks @almolo for raising a seperate thread about Cobertura.

SonarCloudAnalyze@2 fails if code coverage is in Cobertura format

Environment: I have Azure DevOps pipeline and SonarCloud integration. It’s a .NET project. The problem - SonarCloudAnalyze@2 fails to process test coverage information when that information is in *.cobertura.xml format. It only processes the coverage information when it’s in binary .coverage format. Question - is Cobertura format not supported by SonarCloudAnalyze@2, or is the problem somewhere else? First, pipeline code that works (coverage collected in .coverage format): - task: DotNetCore…

And thanks, @dellch, for pointing out the Azure DevOps docs. We don’t maintain these, but it looks like the folks who do update them on a monthly cadence.

[norbert-groen]

Today we updated the Sonar Extension to version 6. It doesn’t use the codecoverage.exe anymore.
So everything is fine now. We donot have to change our pipeline other than the versions of the tools.

Thank you Sonarsource dotnet team!