Number of unit tests is not measured with nested junit 5 tests

metrics
java
tests
sonarqube

(Florian Tanner) #1

Hello everyone

Versions

SonarQube: 6.7.5 lts
Scanner for gradle: 2.6.2
Plugin: SonarJava 5.8

sonarqube gradle configuration:

sonarqube {
   properties {
      property "sonar.projectName", "projectName"
      property "sonar.projectKey", "projectKey"
      property "sonar.host.url", "host"
      property "sonar.scm.disabled", "true"
      property "sonar.java.coveragePlugin", "jacoco"
      property "sonar.junit.reportPaths", ["build/test-results/test"]
      property "sonar.jacoco.reportPaths", "build/reports/jacoco/test.exec"
   }
}

what are you trying to achieve

We are trying to measure the number of executed junit5 tests. Where most of the tests reside in a nested non-static inner class.

@DisplayName("X")
@Tag("x")
class XTest {

  @DisplayName("y")
  @Nested
  class MethodYTest {

    @DisplayName("z")
    @Test
    void shouldZ() {...}
    ...

  }
}

Only one test is found. This test does not use a nested class. Therefore we assume the configuration must be correct.
Also the coverage is gathered successfully.

What is different from the case with nested tests is the naming of the generated report xml-files (and ofc the content).
(The results are taken from different test cases and anonymized in the same manner)

/src/project/sub1/build/test-results/test/TEST-package.XTest.xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="package.XTest" tests="1" skipped="0" failures="0" errors="0" timestamp="time" hostname="host" time="ms">
  <properties/>
  <testcase name="shouldZ()" classname="package.XTest" time="0.001"/>
  <system-out><![CDATA[]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

vs

/src/project/sub1/build/test-results/test/TEST-package.XTest$YTest.xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="package.XTest$YTest" tests="5" skipped="0" failures="0" errors="0" timestamp="time" hostname="host" time="ms">
  <properties/>
  <testcase name="shouldZ()" classname="packageXTest$YTest" time="ms"/>
  <testcase name="shouldZ1()" classname="package.XTest$YTest" time="ms"/>
  <testcase name="shouldZ2()" classname="package.XTest$YTest" time="ms"/>
  <testcase name="shouldZ3()" classname="package.XTest$YTest" time="ms"/>
  <testcase name="shouldZ4()" classname="package.XTest$YTest" time="ms"/>
  <system-out><![CDATA[]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

what have you tried so far to achieve this

. played around with the property “sonar.junit.reportPaths” and the deprecated option (reportPaths)

Has anyone a clue what might be the issue here?

I’ll gladly provide more information if needed.

Regards
Florian


(G Ann Campbell) #2

Hi Florian,

SonarQube analysis just reads your reports, so if the data isn’t want you expect, you should take a look at the data source.

 
Ann