Must-share information (formatted with Markdown):
-
which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
8.2.0 -
what are you trying to achieve
Our organization puts high value on their black box feature testing of a product, which is run separately from unit and integration testing. How can we get SonarQube to show a test execution report separate from, and in addition to, the regular jacoco coverage and Unit Tests data? See screenshot below for an example test execution report. For the Feature Test execution report, coverage is not important/applicable, just the test execution summary data.
- what have you tried so far to achieve this
Note: I’m using Gradle with the java and jacoco plugins. The unit test coverage and unit test data show up fine in SonarQube.
I have tried creating a fake “Generic Execution” report as described in the docs,
Generic Test > Generic Execution
and adding a reference to it via the below reports/path properties (note I tried various combinations of these properties).
It seems the SonarQube gradle task succeed (i.e. task initially failed due to syntax in the generic report file, but no error after correcting that), but the execution report does not show up anywhere in SonarQube.
From the sonar docs:
Language Property Remarks
All sonar.testExecutionReportPaths Comma-delimited list of paths to execution reports in the Generic Execution Data format.
Java sonar.junit.reportPaths Comma-delimited list of paths to Surefire XML-format reports
from: build.gradle
sonarqube {
properties {
property "sonar.projectBaseDir", "${project.rootDir}"
// tried various combinations of these two properties and their values
//property "sonar.junit.reportPaths", ['build/test-results/test','build/fake-reports/generic-execution']
property "sonar.testExecutionReportPaths", ['build/fake-reports/generic-execution/TEST-fake-execution-report.xml']
}
}
My TEST-fake-execution-report.xml created from the documented example:
<testExecutions version="1">
<file path="src/test/java/com/mycomp/somepkg/WorkflowFeatureTest.java">
<testCase name="test1" duration="5"/>
<testCase name="test2" duration="500">
<skipped message="short message">other</skipped>
</testCase>
<testCase name="test3" duration="100">
<failure message="short">stacktrace</failure>
</testCase>
<testCase name="test4" duration="500">
<error message="short">stacktrace</error>
</testCase>
</file>
</testExecutions>
Example of the type of test execution report we would like to see in SonarQube
Note: coverage does not matter for Feature Test execution report, just the summary data.
