Dotnet core trx test report parsing error in azure devops

SonarQube Cloud
1

Writing this up based on Mickaël Caro’s feedback from this post

Doing static code analysis with Azure DevOps. I’ve been trying to experiment with different configurations but coverage is always 0% in sonar cloud

some logs:
SonarScanner for MSBuild 4.6.2
Using the .NET Core version of the Scanner for MSBuild
Pre-processing started.
…
sonar.organization=leasecrunch
sonar.host.url=https://sonarcloud.io/
sonar.projectKey=LeaseCrunch.Core
sonar.projectName=LeaseCrunch.Core
sonar.projectVersion=2.8.0.7
sonar.branch.name=develop
sonar.cs.vscoveragexml.reportsPaths=/home/vsts/work/_temp/**/*.trx
sonar.visualstudio.enable=false
…
22:09:19.338 INFO: Sensor C# Tests Coverage Report Import [csharp]
22:09:19.355 INFO: Parsing the Visual Studio coverage XML report /home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_39.trx
22:09:19.360 WARN: Could not import coverage report ‘/home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_39.trx’
22:09:19.360 INFO: Parsing the Visual Studio coverage XML report /home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_30.trx
22:09:19.361 WARN: Could not import coverage report ‘/home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_30.trx’
22:09:19.361 INFO: Parsing the Visual Studio coverage XML report /home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_35.trx
22:09:19.362 WARN: Could not import coverage report ‘/home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_35.trx’
22:09:19.362 INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=24ms

4

Hi Brian,

It was not intended to discover a slight issue here, but it appears that the exception is not logged when a parsing error occur, so we open up an issue on our side, it will be fixed soon.

In the mean time, i see that you are giving the trx to the sonar.cs.vscoveragexml.reportsPaths which is not right.
The Scanner for MSBuild will detect automatically the *.trx files in the solution and convert them to *.coveragexml files which are understandable by the analyzer to get the coverage of your code.

If you want to provide a custom folder to your *.trx files, please use “sonar.cs.vstest.reportsPaths” instead.

Of if you still want to use sonar.cs.vscoveragexml.reportsPaths, you should provide the path to your already converted *.coveragexml files.

Mickaël

1 Like
5

thank you!

unfortunately, I was only setting sonar.cs.vscoveragexml.reportsPaths because the scanner wasn’t picking up my trx report files without it. here’s some debug output:

dotnet test step:

[command]/opt/hostedtoolcache/dotnet/dotnet test /home/vsts/work/1/s/src/AuditService.sln --logger trx --results-directory /home/vsts/work/_temp --configuration Release
...
Test Run Successful.
Total tests: 52
     Passed: 52
 Total time: 3.5409 Seconds

code analysis step:

SonarScanner for MSBuild 4.6.2
Using the .NET Core version of the Scanner for MSBuild
Post-processing started.
...
sonar.organization=leasecrunch
sonar.host.url=https://sonarcloud.io/
sonar.projectKey=LeaseCrunch.AuditService
sonar.projectName=LeaseCrunch.AuditService
sonar.projectVersion=1.0.3.5
sonar.branch.name=refs/heads/develop
sonar.visualstudio.enable=false

sonar.modules=2BFAA169-F39B-4370-8F8D-5FACA2AC8BAC,8D8BE766-AF3E-4B4D-9493-5FD74D052EE3,B39656B5-FEF8-4902-9B25-983B9FD570E9
...
Calling the SonarQube Scanner...
Executing file /home/vsts/work/_tasks/SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255/1.7.0/dotnet-sonar-scanner-msbuild/sonar-scanner-3.3.0.1492/bin/sonar-scanner
  Args: -Dsonar.scanAllFiles=true -Dproject.settings=/home/vsts/work/1/.sonarqube/out/sonar-project.properties --embedded --debug <sensitive data removed>
  Working directory: /home/vsts/work/1
  Timeout (ms):-1
  Process id: 4427

I don’t see any debug output like Sensor C# Tests Coverage Report Import [csharp] or trx anywhere - not sure what to look for so I attached the whole logcode_analysis_debug.txt (131.1 KB)

6

Where did you set the sonar.cs.vscoveragexml.reportsPaths property ?

7

I removed the setting, so I’m not setting it anymore as I thought you suggested. I don’t have any requirements to load my coverage report from a specific location. I was just setting that trying to get it working.

8

Have you tried setting this one : sonar.cs.vstest.reportsPaths with the path where your trx are located ?

I don’t see any lookup of test file in the logs, are you executing the test task before the “Run Analysis” one ?

9

I will try the sonar.cs.vstest.reportsPaths setting.

Yes, I am running the “Run Code Analysis” after my step that performs dotnet test

10

after setting sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)/**/*.trx I now see this output:

17:57:28.559 INFO: Sensor C# Unit Test Results Import [csharp]
17:57:28.578 INFO: Parsing the Visual Studio Test Results file /home/vsts/work/_temp/_fv-az644_2019-07-30_17_56_47.trx
17:57:28.594 INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=35ms

however, in sonar, while it does now report that I have 52 tests, coverage is still 0%

11

Thank you.

I will do some tests on my side and let you know.

12

Hi Brian,

After having done further tests, here is the conclusion

  • Basic ‘TRX’ files from dotnet test only contains tests results, not coverage, this is the reason why it says that the parsing is in progress, but without any coverage shown on SonarCloud.
  • To enable code coverage in trx files, you have to provide ’ --collect “Code Coverage” ’ option to the dotnet test command in the dotnet task of your build pipeline; the problem is that currently code coverage collection is not supported on linux, only on windows (see https://github.com/microsoft/vstest/issues/981 )

They are few workaround possible

  • Either use a windows agent to benefit from the dotnet test experience
  • Or there are few open source tools to have coverage, but it requires some changes on your solution (see https://github.com/tonerdo/coverlet )

Let us know if you ended up finding a good solution for you.

Thanks !

1 Like
13

Switching over to a windows agent (Hosted Windows 2019 with VS2019 in this case) and adding --collect “Code Coverage” resolved this. thank you!