Load Multiple Report files in Sonarqube

SonarQube
,
1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    Community Edition Version 9.5 (build 56709)
    sonar-scanner-4.7.0.2747

  • what are you trying to achieve
    Hello, I have a set o xunit tests in my .NET project and also a set of cypress tests.
    I can successfully load the xunit tests report using the parameter: sonar.cs.xunit.reportsPaths
    I can also successfully load the cypress tests report using the parameter: sonar.testExecutionReportPaths
    But I can’t load both at the same time. If l add both parameters, I only see the results of the second one. How can I tell the sonar scanner to use all tests reports provided?

  • what have you tried so far to achieve this

4

Hello @ffiuza

Sorry for the delay.

Are you using the Scanner for .NET? What is the version of the Scanner for .NET (MSBuild) that you are using?

To be honest, we’ve never had this scenario in mind. I’ll ask internally.

Meanwhile, could you give us the verbose logs of the END step, so that we can use that information during our investigation?

You need to pass a parameter during the Begin step in order to enable verbose mode.

SonarScanner.MSBuild.exe begin /k:“MyProject” /d:sonar.verbose=true

or

      - task: SonarQubePrepare@1
        displayName: 'Code Analysis - Begin'
        inputs:
          ...
          extraProperties: |
            sonar.verbose=true

Thanks!

6

Hello @Andrei_Epure

We are using MSBuild 5.7.

This is the log generated with verbose=true. I removed some lines related to ours project file name, but left all lines related to the testing part. As you can see both paths have been analyzed successfully, but when I go to the Sonar Dashboard, the results for the unit tests were not pushed. In this case, I used the these properties:

/d:sonar.cs.vstest.reportsPaths=.\path_to\**\TestOutput.trx
/d:sonar.cs.opencover.reportsPaths=.\path_to\*\*\coverage.opencover.xml
/d:sonar.testExecutionReportPaths=..\path_to\dist\cypress-sonarqube-reports.all.xml

Coverage were pushed, but sonar.cs.vstest.reportsPaths or sonar.cs.xunit.reportsPaths are not pushed if I use sonar.testExecutionReportPaths at the same time.

SonarScanner for MSBuild 5.7

Using the .NET Framework version of the Scanner for MSBuild

Post-processing started.

18:09:46.724 INFO: Sensor Generic Test Executions Report

18:09:46.724 INFO: Parsing C:\path_to\cypress-sonarqube-reports.all.xml

18:09:46.758 INFO: Imported test execution data for 24 files

18:09:46.758 INFO: Sensor Generic Test Executions Report (done) | time=34ms

18:09:46.802 DEBUG: Sensors : C# -> Analysis Warnings import -> C# Tests Coverage Report Import -> C# Unit Test Results Import -> Zero Coverage Sensor

18:09:50.215 INFO: Found 12 MSBuild C# projects: 11 MAIN projects. 1 TEST project.

18:09:50.215 INFO: Sensor C# [csharp] (done) | time=3413ms

18:09:50.215 INFO: Sensor Analysis Warnings import [csharp]

18:09:50.216 DEBUG: Searching for analysis warnings in C:\path_to\.sonarqube\out

18:09:50.221 INFO: Sensor Analysis Warnings import [csharp] (done) | time=6ms

18:09:50.221 INFO: Sensor C# Tests Coverage Report Import [csharp]

18:09:50.222 DEBUG: Analyzing coverage with wildcardPatternFileProvider with base dir 'C:\path_to\.' and file separator '\'.

18:09:50.222 DEBUG: Pattern matcher extracted prefix/absolute path 'C:\path_to\Tests' from the given pattern '.\Tests\*\*\coverage.opencover.xml'.

18:09:50.224 DEBUG: Gathering files for wildcardPattern '*\*\coverage.opencover.xml'.

18:09:50.252 DEBUG: Pattern matcher returns '1' files.

18:09:50.253 DEBUG: The current user dir is 'C:\path_to'.

18:09:50.253 INFO: Parsing the OpenCover report C:\path_to\Tests\TestResults\01ce9939-8d54-4e1e-909b-33338450d918\coverage.opencover.xml

18:09:50.742 INFO: Adding this code coverage report to the cache for later reuse: C:\path_to\.\.\Tests\TestResults\01ce9939-8d54-4e1e-909b-33338450d918\coverage.opencover.xml

18:09:50.744 DEBUG: Analyzing coverage after aggregate found '499' coverage files.

18:09:51.115 DEBUG: The total number of file count statistics is '499'.

18:09:51.115 INFO: Coverage Report Statistics: 499 files, 499 main files, 499 main files with coverage, 0 test files, 0 project excluded files, 0 other language files.

18:09:51.115 INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=894ms

18:09:51.115 INFO: Sensor C# Unit Test Results Import [csharp]

18:09:51.116 DEBUG: Pattern matcher extracted prefix/absolute path 'C:\path_to\Tests' from the given pattern '.\Tests\**\TestOutput.trx'.

18:09:51.116 DEBUG: Gathering files for wildcardPattern '**\TestOutput.trx'.

18:09:51.129 DEBUG: Pattern matcher returns '1' files.

18:09:51.129 DEBUG: The current user dir is 'C:\path_to'.

18:09:51.129 INFO: Parsing the Visual Studio Test Results file 'C:\path_to\TestOutput.trx'.

18:09:51.131 DEBUG: Parsed Visual Studio Test Times - duration: 6210.

18:09:51.132 DEBUG: Parsed Visual Studio Test Counters - total: 19, failed: 4, errors: 0, timeout: 0, aborted: 0, executed: 19.

18:09:51.137 INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=22ms

18:09:51.137 INFO: Sensor Zero Coverage Sensor

18:09:51.217 INFO: Sensor Zero Coverage Sensor (done) | time=80ms

18:09:51.980 DEBUG: SCM revision ID '419de89c1d5de49789ac921b31a27803d286a6f8'

18:09:53.129 INFO: Analysis report generated in 707ms, dir size=3.8 MB

18:09:55.153 INFO: Analysis report compressed in 2023ms, zip size=2.2 MB

18:09:55.153 DEBUG: Upload report

18:09:55.193 DEBUG: POST 200 http://localhost:9000/api/ce/submit?projectKey=id | time=38ms

18:09:55.196 INFO: Analysis report uploaded in 43ms

18:09:55.198 INFO: ANALYSIS SUCCESSFUL, you can find the results at: http://localhost:9000/dashboard?id=id

18:09:55.198 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report

18:09:55.198 INFO: More about the report processing at http://localhost:9000/api/ce/task?id=AYICn6zrdIZr0gro5j5p

18:09:55.201 DEBUG: Post-jobs :

18:09:55.205 DEBUG: eslint-bridge server will shutdown

18:10:02.367 INFO: Analysis total time: 29.782 s

18:10:02.369 INFO: ------------------------------------------------------------------------

18:10:02.369 INFO: EXECUTION SUCCESS

18:10:02.369 INFO: ------------------------------------------------------------------------

18:10:02.369 INFO: Total time: 30.725s

18:10:02.423 INFO: Final Memory: 16M/64M

18:10:02.423 INFO: ------------------------------------------------------------------------

Process returned exit code 0

The SonarScanner CLI has finished

18:10:02.967 Post-processing succeeded.
7

Just to understand if we need to support a new feature - what is the scenario you are in, having two separate test reports?

8

Hello Andrei, yes that’s the scenario, 2 separate test reports. I think this feature is already there because I can load both test reports if they share the same sonar property, for instance this works:

/d:sonar.testExecutionReportPaths=.\report1.xml,.\report2.xml

But this does not:

/d:sonar.testExecutionReportPaths=.\report1.xml
/d:sonar.cs.xunit.reportsPaths=.\report2.xml

My workaround was to convert one of the reports so they are sharing the same format.

9

Hello Francisco

Yes, we support multiple test and code coverage report paths, as long as they share the same format. We then aggregate the data.

Indeed, this is not a supported case currently in the product.

12

So, if this is really the case, then Sonar should emit a warning saying which report will be ignored, and not say it processed both files but then just used one. By just reading the log messages, that says both files were processed, one would assume both would be considered.

Please update the Sonar Documentation and the log messages to make it clear only one report format can be used.

1 Like
13

Thanks for the feedback @ffiuza . I have raised it to Product Management.

1 Like
14

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.