The error "generic test execution report '/github/workspace/test/junit.xml'"

SonarQube Cloud
,
1

The error “generic test execution report ‘/github/workspace/test/junit.xml’”

2

Hey there.

sonar.testExecutionReportPaths accepts reports in the generic execution data format. If your test execution tool doesn’t offer a compatible reporter out of the box, you’ll have to build a converter.

4

To be clear, there’s no conversion required for coverage results that you pass to sonar.typescript.lcov.reportPaths (although please be careful to use the correct, non-deprecated sonar.javascript.lcov.reportPath).

sonar.testExecutionReportPath is used to import test execution data (how many unit tests were run). This is very optional and doesn’t add a lot of value to your analysis results (I would say almost none). You can write a converter from Junit to the generic format, transforming the XML, but we don’t have one to offer.

6

To show test execution results. This is not coverage. You can check to see if the coverage import is working either by… looking at the coverage figure, out looking at the logs:

7

The generic test execution format seems to follow the Common JUnit XML Format (see here junitxml), but does it also implement properties for testCase elements (see here properties) ?

8

I mean… they are both XML, but they are very different formats if you look at the XML of each example.

Generic Test Data:

<testExecutions version="1">
  <file path="testx/ClassOneTest.xoo">
    <testCase name="test1" duration="5"/>
    <testCase name="test2" duration="500">
      <skipped message="short message">other</skipped>
    </testCase>
    <testCase name="test3" duration="100">
      <failure message="short">stacktrace</failure>
    </testCase>
    <testCase name="test4" duration="500">
      <error message="short">stacktrace</error>
    </testCase>
  </file>
</testExecutions>

JUnit XML:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites time="15.682687">
    <testsuite name="Tests.Registration" time="6.605871">
        <testcase name="testCase1" classname="Tests.Registration" time="2.113871" />
        <testcase name="testCase2" classname="Tests.Registration" time="1.051" />
        <testcase name="testCase3" classname="Tests.Registration" time="3.441" />
    </testsuite>
    <testsuite name="Tests.Authentication" time="9.076816">
        <testsuite name="Tests.Authentication.Login" time="4.356">
            <testcase name="testCase4" classname="Tests.Authentication.Login" time="2.244" />
            <testcase name="testCase5" classname="Tests.Authentication.Login" time="0.781" />
            <testcase name="testCase6" classname="Tests.Authentication.Login" time="1.331" />
        </testsuite>
        <testcase name="testCase7" classname="Tests.Authentication" time="2.508" />
        <testcase name="testCase8" classname="Tests.Authentication" time="1.230816" />
        <testcase name="testCase9" classname="Tests.Authentication" time="0.982">
            <failure message="Assertion error message" type="AssertionError">
                <!-- Call stack printed here -->
            </failure>            
        </testcase>
    </testsuite>
</testsuites>

The latter doesn’t include any file data, and the former doesn’t mention testSuites.

9

This means we cannot upload test reports generated by pytest, or at least we need some kind of converter to reformat the report before.

1 Like
10

For Python we support xUnit out of the box, but if that’s not an option, then like I said:

11

pytest is a widely used standard tool. SonarQube should be able to process its output directly without requiring users to perform any additional conversions.