GenericTestExecution does not work with maven multi-modules

Using Sonarqube 7.7 with latest sonar-maven-plugin

Having a Maven multi-module structure like:

a with property (<sonar.testExecutionReportPaths>${project.build.directory}/karma-reports/TEST-gui.Sonar.xml</sonar.testExecutionReportPaths>)

  • b (this module contains the generic test execution report)
  • c

Executing sonar:sonar on parent module a does not execute the sensor on module b to collect the generic test execution.
Executing sonar:sonar directly on module b works as expected.

Kind regards,
Michael

4 Likes

Sorry for the late reply, Michael.
Do you happen to have a simple reproducer? Or do you have logs with debug enabled of the execution of module a?

I tested this now with SonarQube 7.9 LTS and maven sonar:sonar goal

The problem is that the sensor for the generic test reports is only checked at the top maven project, not on its child projects.

This is very important for us to work, please consider allowing generic test reports for all maven child modules.

Hi Michael,

The request for either logs or a simple reproducer stands. Do you have either of those available?

 
:slight_smile:
Ann

There is not much outcome of the logs as the sensor only runs on the top level module as I described. I think it should be straight forward for you to change the execution policy of this sensor to be also executed on submodules.
If I made the wrong conclusion out of the logs please let me know, then I will prepare a reproducer.

Hi

I’ll let Duarte decide about the logs, but it makes sense to go ahead with a reproducer.

 
Thx,
Ann

I confirm that this problem happens with version 7.9.1. The sonar.testExecutionReportPaths property is ignored when specified in the pom of a submodule. The log miss the any reference to “Sensor Generic Test Executions Report”.

The property is taken in consideration when included in the main module’s pom, an the logs contains this part:

11:15:51  [INFO] ------------- Run sensors on module ***<main module name>***
11:15:51  [INFO] Sensor JaCoCo XML Report Importer [jacoco]
11:15:51  [INFO] Sensor JaCoCo XML Report Importer [jacoco] (done) | time=0ms
11:15:51  [INFO] Sensor HTML [web]
11:15:51  [INFO] Sensor HTML [web] (done) | time=0ms
11:15:51  [INFO] Sensor XML Sensor [xml]
11:15:51  [INFO] 1 source files to be analyzed
11:15:51  [INFO] Sensor XML Sensor [xml] (done) | time=13ms
11:15:51  [INFO] Sensor Generic Test Executions Report
11:15:51  [INFO] Parsing /opt/jenkins/workspace/***main module name****/***submodule name****/target/test/javascript/reports/karma/ut_report.xml
11:15:51  [INFO] 1/1 source files have been analyzed
11:15:51  [INFO] Imported test execution data for 0 files
11:15:51  [INFO] Test execution data ignored for 1 unknown files, including:
11:15:51  src/app/app.component.spec.ts
11:15:51  [INFO] Sensor Generic Test Executions Report (done) | time=6ms

As expected the content of the report is ignored because the main module doesn’t cotains the tested files.

Thanks for your confirmation. It’s really disappointing that this feature does not work at all.
Any estimation when this will be fixed?
Is there already an issue ticket created which can be linked here?

Looks like this is broken since 7.6
https://jira.sonarsource.com/browse/SONAR-11530

Hi,

Thanks @reitzmichnicht for the detailed analysis of the issue.
I have the exact same issue where I have a multi-module application made of React modules.
I can generate an individual Generic Test Data report per module using jest-sonar-reporter in each module.
But how can we make SonarQube 7.6+ take these into account ? Any hint here would be welcome.

Thanks,
Nico

@ganncamp any updates on this? Looks like this is a regression introduced in 7.6 which could be easily fixed.

As Andrea said, the sonar.testExecutionReportPaths is simply ignored when specified in the pom of a module.

The only way to trigger the execution of the Generic Test Execution Sensor is to specify the property in the root pom. However, although the sensor executes, it will complain about not being able to find the source files referenced in the reports. Reports are generated by child modules and often reference source files relative to their own base - such relative reference cannot be resolved from the root pom - hence the reason why the Sensor complains (executed from the root).

Although there is a way to trigger the execution of the Sensor - this is not a viable workaround since it will ultimately refuse to consider the content of the read reports :frowning:

The solution is even more confusing that we can specify properties for other sensors in child poms without any problem. Please, have a look at this issue…

BTW: I confirm the issue is still there with SonarQube 8.2.

@reitzmichnicht Did you find any viable workaround for this multi-module setup ?
@ganncamp What’s the status about this issue? Any chance you’ll be looking at it in the near future?

BTW, the specs for the Generic Test Execution Report format (https://docs.sonarqube.org/latest/analysis/generic-test/) says:

Insert a file element for each test file. Its path attribute can be either absolute or relative to the root of the module.

As we have seen here, this is definitely not correct… paths must either be:

  • absolute;
  • or relative to the root of the project (i.e. in case of maven multi-module, the path to the project pom)

But certainly not relative to the root of the module!

Sorry, no updates on my side.

 
Ann

Sadly I did not find any workaround yet for this problem.

@ganncamp A year already this issue is opened and still no updates :frowning:

@reitzmichnicht We decided to make our own Sonar Sensor plugin to upload Generic Test Execution reports…

I was able to work around this in my setup but I don’t like it as it is ugly as hell…

The basic steps are :

  • Produce the Generic Test Execution Reports with absolute file names
  • In the root pom.xml, the one used to launch the SonarQube analysis, add a Maven property called sonar.testExecutionReportPaths
  • In the sonar.testExecutionReportPaths, enter the paths to each individual Generic Test Execution Reports files, separated by a comma.These paths can be relative to the root pom.xml. For example, <sonar.testExecutionReportPaths> moduleA/target/generic-test-report.xml,moduleB/target/generic-test-report.xml</sonar.testExecutionReportPaths>

As I was saying, it is ugly as hell because :

  • You have to change the way the Generic Test Execution Reports are generated to use absolute file names, and this is not always possible if you rely on a 3rd party plugin for example
  • You have to manually list all your modules in the property instead of relying on Maven information. This is error prone in case a new module is added for example, it’s easy to add it in Maven <modules> section and forget to update the property.
  • You have to introduce this property in the root pom.xml file. My personal style is to keep this file as clean as possible and use a dedicated pom.xml for plugins configuration
1 Like

@ganncamp I’m not really satisfied about the product roadmap here.

We are a paying sonarqube customer and this bug is still open on 8.5 we use. That limits the way how we are able to use sonar.
I even do not see a jira ticket created for this issue!