Need help for Unit Test Code coverage for SonarCloud

SonarQube Cloud
,
1

Hi Team,
I am working on dotnet project and I was able to successfully generate sonar cloud scan analysis report from Azure DevOps by using the Sonar Cloud Pugin. However, the Coverage and Unit Tests section shows 0.0%. Could you help me as to what would be required to include the unit test coverage results?

2

Welcome, @palak.joshi!

How are you triggering your unit test and code coverage files? You need to make sure of the following:

  1. Are you using VSTest, NUnit, or xUnit for unit tests? If so, make sure you create the *.trx or *.xml file and set its path for the associated sonar property. See “Import unit test results” section in [Coverage & Test Data] Generate Reports for C#, VB.net.
  2. Are you using Visual Studio CodeCoverage, dotCover, OpenCover, or Coverlet? If so, make sure to generate the code coverage file first then set its path to the associated sonar property. See “Import test coverage reports” in [Coverage & Test Data] Generate Reports for C#, VB.net.
  3. Make sure to add these 2 sonar properties (one for unit tests, one for code coverage) to the MSBuild begin step. Then you can proceed with the build step and the end step.

Here is where I got the instructions:

Here is an example I ran with VSTest and Visual Studio Code Coverage:

# run unit tests and save results
$ vstest.console.exe /Logger:trx "SomeConsoleApplicationTest\bin\Debug\SomeConsoleApplicationTest.dll"

# generate code coverage
$ vstest.console.exe --EnableCodeCoverage "SomeConsoleApplicationTest\bin\Debug\SomeConsoleApplicationTest.dll"

# covert the binary .coverage file into .coveragexml
$ CodeCoverage.exe analyze /output:"C:\Users\jojo\Projects\sonar-scanning-examples\sonarqube-scanner-msbuild\CSharpProject\TestResults\dee3f166-f02a-432c-8b4d-3abca4467bb5\jojo_joeTingsanchali 2020-10-14 19_02_26.coveragexml" "C:\Users\jojo\Projects\sonar-scanning-examples\sonarqube-scanner-msbuild\CSharpProject\TestResults\dee3f166-f02a-432c-8b4d-3abca4467bb5\jojo_joeTingsanchali 2020-10-14 19_02_26.coverage"

# Begin step with the vstest (for unit tests) and vscoveragexml (for code coverage) parameters
$ SonarScanner.MSBuild.exe begin -k:"CSharpProjectExample" -d:"sonar.host.url=http://localhost:9000" -d:"sonar.cs.vstest.reportsPaths=**/*.trx" -d:"sonar.verbose=true"-d:"sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml"

# Rebuild
$ MSBuild.exe ./SomeConsoleApplication.sln -t:Rebuild

# End
$ SonarScanner.MSBuild.exe end

If this is confusing or these instructions don’t work, please provide the commands you used and the scanner log output from your commands including using -d:"sonar.verbose=true" (or /d) in the MSBuild begin step.

Joe

1 Like
3

Hi @Joe_Tingsanchali , thanks for getting back. By the begin step, do you mean “Prepare Analysis on SonarCloud step”?

I added the sonar properties as:

sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml
sonar.cs.vstest.reportsPaths=**/*.trx
sonar.debug=true
sonar.verbose=true

and Please find below the Agent information below for running the pipeline:
image

Please find below the log for the Run Code Analysis step.SonarCloud Log codecoverage.txt (5.9 KB)

Could you please help me if I’m missing something?

4

HI, @Joe awaiting your response on the above commands and what extra I need to add.

5

Hello @palak.joshi,

I apologize for my delayed response. Thank you for the log, it looks like it found the .coverage file and converted it correctly into the .coveragexml file. Can you confirm there is something inside both files (i.e. make sure it’s not blank or empty)? Can you show the logs for the entire build (MSBuild.exe begin, rebuild. MSBuild.exe end steps, etc.)? It’s very strange that no other logs are shown in your attachment for the SonarCloud step.

Here’s an example for you using VSTest and VSCoverageXML that worked for me:

  • Note the extraProperties I have listed for the SonarCloudPrepare@1 (“Prepare Analysis for SonarCloud” task). Try referencing the $(Agent.TempDirectory), although it looks like your log show its defaulted to the correct base directory already.
trigger:
- master

pool:
  vmImage: 'windows-latest'

steps:
- task: VisualStudioTestPlatformInstaller@1
  inputs:
    packageFeedSelector: 'nugetOrg'
    versionSelector: 'latestPreRelease'
- task: NuGetCommand@2
  inputs:
    command: 'restore'
    restoreSolution: '**/*.sln'
    feedsToUse: 'select'
- task: SonarCloudPrepare@1
  inputs:
    SonarCloud: 'sonarcloud'
    organization: 'joetingsanchali'
    scannerMode: 'MSBuild'
    projectKey: 'joetingsanchali_sonar-scanning-someconsoleapp'
    projectName: 'sonar-scanning-someconsoleapp'
    extraProperties: |
      sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)\**\*.trx
      sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)\**\*.coveragexml
      sonar.verbose=true
- task: MSBuild@1
  inputs:
    solution: '**\*.sln'
    msbuildArguments: '-t:Rebuild'
- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*test*.dll
      !**\*TestAdapter.dll
      !**\obj\**
    codeCoverageEnabled: true
    searchFolder: '$(System.DefaultWorkingDirectory)'
- task: SonarCloudAnalyze@1
- task: SonarCloudPublish@1
  inputs:
    pollingTimeoutSec: '300'

By the way, you can remove sonar.debug=true since that doesn’t do anything.

Joe

6

@Joe Your documentation really only covers Windows. It has a section that says something akin to “notice that we don’t need to do anything special because this is on windows” - this is not helpful to those of us who need to test our apps outside of windows. The other info I’ve found from admins on this forum use the VSTest step which is windows-only. The documentation also mentions that codecoverage.exe can be used to convert the .coverage file to xml. However, that seems to be windows-only as well. I’m going through so many different roads that all seem to lead to “sonar can’t be used properly for code coverage without a windows dependency”. Is that true? Please let me know now and we can just roll our own process with cobertura reports or something. We pay for this tool and yet the support system is so limited.

edit: I’ve since figured something out by installing dotnet-coverage (guess this is the new cross platform dotnet tool replacing codecoverage.exe) and using the merge command. But it was painfully tedious and not obvious at all. Is there any way you could add supporting documentation as this appears to be the most straightforward way that works with sonarcloud? Cobertura, etc. aren’t supported formats.

7

Hey @SD321

The SonarCloud documentation on .NET Code Coverage covers the dotnet-coverage tool.

8

I… I got nothing. I usually have some pretty good googling skills but I never ended up there. All the links from here to the page that shows up in the debug logs when no coverage file is found didn’t help me end up there. I probably got hung up on 5 pages I kept coming back to somehow. Thank you so much @Colin and sorry for the inconvenience. User error :sweat_smile:

1 Like
9

No worries.

Admittedly, on our side, neither the log messages you would have seen in the scanner logs or the [Coverage] Troubleshooting guide for .NET code coverage import point to these (relatively new) SonarCloud docs.

Maybe we can do something better to get users to the right place. I’ll ping around.

1 Like