Test kpi import for pure test projects (no classes)

Henning_Luther · March 13, 2023, 8:52pm

We have a framework, where we configure tests into the testng-api where mostly the test is not connected to any testclass(but just a wrapper to work with testng or junit). We use a pipeline calling “mvn sonar:sonar”. As failsafe/surefire generates testng-output and junit-output we expected it to work out of the box, but the metrics didnt show up. As we digged deeper we saw the org/sonar/plugins/surefire/SurefireJavaParser tries to lookup the Java-class mentioned in the testreport(classname=“TheTestClass”) and when not found can’t generate the metrics. Since our tests have no Java(Test)Class in the project but a wrapper from the testng-bridging dependency, this will never match. So my question is: Is there a way to transmit the test metrics with another scanner or over an api-call or what might be the easiest way to make it work?

ganncamp · March 14, 2023, 5:53pm

Hi,

Welcome to the community!

To be clear, you’re generating a test coverage report, but it’s not being used/honored? And you believe that’s because there are no underlying test classes?

Ann

Henning_Luther · March 14, 2023, 6:00pm

The coverage is generated by jacoco, this is working. On top i want to have the Test-count, Fail-ratio and Test-runtime which gets pulled from JUnitformat. The Scanner wants to mach the classname. The testproject itself has no Classes to refer to.

github.com

SonarSource/sonar-java/blob/f631e568f9b6a60c1d4296e22a0782e30b730d15/java-surefire/src/main/java/org/sonar/plugins/surefire/SurefireJavaParser.java#L137


      
              if (report.getTests() > 0) {
                negativeTimeTestNumber += report.getNegativeTimeTestNumber();
                save(report, entry.getKey(), context);
              }
            }
            if (negativeTimeTestNumber > 0) {
              LOGGER.warn("There is {} test(s) reported with negative time by surefire, total duration may not be accurate.", negativeTimeTestNumber);
            }
          }
          
          
private Map<InputFile, UnitTestClassReport> mapToInputFile(Map<String, UnitTestClassReport> indexByClassname) {
            Map<InputFile, UnitTestClassReport> result = new HashMap<>();
            indexByClassname.forEach((className, index) -> {
              InputFile resource = getUnitTestResource(className, index);
              if (resource != null) {
                UnitTestClassReport report = result.computeIfAbsent(resource, r -> new UnitTestClassReport());
                // in case of repeated/parameterized tests (JUnit 5.x) we may end up with tests having the same name
                index.getResults().forEach(report::add);
              } else {
                LOGGER.debug("Resource not found: {}", className);
              }

ganncamp · March 14, 2023, 6:02pm

Hi,

Sorry, but the only way to work would be to generate the tests to file.

Ann

Henning_Luther · March 14, 2023, 6:05pm

We have the Unit files. The problem is the class mapping, cause without it it doesnt work. The classname in the Test must be mappable to a class resource in the project. But we dont have a class there

Henning_Luther · March 14, 2023, 6:07pm

You mean to generate Java files out of the configs. might be a workaround to just generate one