Sonar Scanner is crashing when the javascript execution tests report is missing

SonarQube Server / Community Build
1

Hello,

SonarScanner version: 4.5.0.2216
SonarQube version: 8.6.0.39681

We’re using the sonar scanner on a JavaScript project with coverage (sonar.javascript.lcov.reportPaths) and tests execution reports (sonar.testExecutionReportPaths as specified in the Sonar Scanner Documentation) analysis.

When the execution report is missing, because you’ve decided to skip the tests execution in your ci pipeline for instance, the sonar scanner is crashing with the following error:

# sonar-scanner -Dsonar.projectKey=Project -Dsonar.projectName=Project -Dsonar.projectVersion=1.0 -Dsonar.sourceEncoding=UTF-8 -Dsonar.forceAnalysis=true -Dsonar.projectBaseDir=. -Dsonar.sources=web/themes/custom/project/src/js,web/modules/custom/ -Dsonar.exclusions=**/*.spec.js,**/*Test.php,**/*.ndjson,**/*.png,**/*.csv -Dsonar.coverage.exclusions=web/modules/custom/**/*.js -Dsonar.tests=web/themes/custom/project/src/tests/,web/modules/custom/project_api/tests/,web/modules/custom/project_static/tests/,web/modules/custom/project_customization/tests/ -Dsonar.php.tests.reportPath=phpunit-report.xml -Dsonar.php.coverage.reportPaths=phpunit-coverage-clover.xml -Dsonar.javascript.lcov.reportPaths=lcov.info -Dsonar.testExecutionReportPaths=xunit-sonar.xml
INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: NONE
INFO: SonarScanner 4.5.0.2216
INFO: Java 11.0.6 AdoptOpenJDK (64-bit)
INFO: Linux 4.19.128-microsoft-standard amd64
...
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 7:43.376s
INFO: Final Memory: 22M/84M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
ERROR: Error during parsing of generic test execution report '/src/xunit-sonar.xml'. Look at the SonarQube documentation to know the expected XML format.
ERROR: Caused by: xunit-sonar.xml (No such file or directory)
ERROR:
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

This project does also contains php code. When the php execution report is missing, the sonar scanner handle the scenario gracefully by just printing the following warning message:

WARN: PHPUnit xml test report not found: phpunit-report.xml

And the scan is not crashing… And it’s the same for a java project with the sonar.coverage.jacoco.xmlReportPaths option.

Is it possible to have the same behavior for the sonar.testExecutionReportPaths option than the language specific options ?

Regards

3

Hello @jhu,

Welcome to the SonarSource community. :wave:. I hope you’ll enjoy it.

Indeed I was able to reproduce your problem, and I agree it looks like a bug. I’ll report internally.
That said:

  • As you may have noticed from the evolution of our product we attach much more importance to code coverage compared to simple unit tests execution reports, because the former measures much better how well you test your code than the latter… This certainly explains why we have a more robust behaviour on test coverage than unit test execution
  • You can workaround the problem quite easily:
    – Either remove the property sonar.testExecutionReportPaths when you don’t run your unit tests
    – Or have a static report file ready (static 0 test run) that you present when you don’t run the tests

To finish, I must say that it’s not very aligned with what we consider best practices to run the pipelines differently from one run to another. How can you verify that at any run you meet the criteria defined if your quality gate if the run context is different ? That seems a wrong pattern to me

Olivier

4

Hello @OlivierK,

Thanks for the reply and for the reporting of the bug. :slight_smile:

I’ll try to apply one of your workaround.

To explain you more why this is a problem for us : We have this option, which is only available for the developer’s testing environment related pipeline, which allow to bypass the execution of the tests.

This is useful when our developers are working on big breaking changes. This option is of course not available for the real build which MUST execute tests etc… :slight_smile:

Regards,
Julien