Sonar issues in parallel Maven build

SonarQube
, ,
1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    • SonarQube: Developer Edition Version 7.9.1 (build 27448)
    • sonar maven plugin: 3.7.0.1746:sonar
  • what are you trying to achieve
    • I am aware that we can’t run the sonar goal through maven when running parallel builds, so we have tried to add a custom profile to the verify stage of our build flow that includes the sonar goal. This expectation is that the parallel build will happen first (during the package phase) and then sonar will run as part of the verify stage after, thus avoiding the parallel problem. Below in the what I have tried I will show the pom snippet as well as the command being run. The end result of this, is that a large number of our modules end up getting skipped, with the claim being that they have been “banned” due to a previous failure, but there are no failures reported that I can find. If I run the entire build sequentially, we do not have this error.
  • what have you tried so far to achieve this
    • pom.xml snippet:
<profiles>
        <profile>
            <id>sonar</id>
            <activation>
                <activeByDefault>false</activeByDefault>
            </activation>

            <build>
                <plugins>
                    <plugin>
                        <groupId>org.sonarsource.scanner.maven</groupId>
                        <artifactId>sonar-maven-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>run-sonar</id>
                                <phase>verify</phase>
                                <goals>
                                    <goal>sonar</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
       </profiles>
  • Command run: mvn -T 1C -B verify -Psonar
  • errors returned:
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.7.0.1746:sonar (run-sonar) on project MY-PROJECT: Please provide compiled classes of your project with sonar.java.binaries property -> [Help 1]

00:07:27.973  [BuilderThread 0] INFO org.apache.maven.cli.event.ExecutionEventLogger - Skipping MY SUB MODULE PROJECT
00:07:27.973  [BuilderThread 0] INFO org.apache.maven.cli.event.ExecutionEventLogger - This project has been banned from the build due to previous failures.
00:07:27.973  [BuilderThread 0] INFO org.apache.maven.cli.event.ExecutionEventLogger - ------------------------------------------------------------------------
2

So I guess the key question here is: is it possible to have a parallel maven build that then runs Sonar as the last “step” of the full build so that it runs serially? If so, what does that need to look like in the pom or on the command line? Right now I am at a loss and feel like I will need to instead use the standalone sonar scanner. I would prefer to have it baked into maven, since I get a lot of things for free that I would need to specify in the sonar-project.properties file for a standalone scanner.

3

Hi,

I’m no Mavenista so it’s not clear to me what the output / outcome of a parallel build is. But I find this error interesting:

Does ‘parallel’ mean that the classes are scattered in multiple locations? Because you can’t analyze Java without providing the classes.

 
Ann

4

Good question Ann! I spent a bunch of time trying to figure this one out and I am not entirely proficient in Maven either, however this appears to be more of a symptom of the underlying problem, which is that some of my modules are getting skipped due to “build failures” that I can’t find anywhere in the logs. From what I understand the flow of the parallel build should be: unit test -> build -> verify. Each of these stages requires the previous stage to have finished before starting the next stage, even in a parallel build. So the unit tests all run in parallel then the packages all build in parallel (provided the build plan can run them all in parallel, since some of the modules depend on others, but that is all defined in the poms) and then verify “runs in parallel”, but the only thing that runs during the verify stage is sonar, so it shouldn’t be doing anything else in parallel.

I think the problem that I need insight on is why a number of my modules are being skipped due to “previous failures”, but only when I include the sonar profile (they don’t get skipped if i run this maven command: mvn -T 1C -B verify).

5

Something else I noticed, is that it looks like Sonar is actually starting instrumentation before the build is actually done, which seems strange, but would also potentially explain the errors. But now the question comes back to maven and why it is getting triggered before the build phase finishes (which I am not certain I have enough maven or sonar plugin for maven knowledge to answer that:
Notice sonar on thread 0 and a builder triggering for a module on thread 1. I would assume that sonar would not even kick off until all the build threads are finished, based on how the pom is setup, right?

00:07:05.703  [BuilderThread 0] INFO org.sonarsource.scanner.maven.SonarQubeMojo - Load global settings (done) | time=159ms
00:07:05.705  [BuilderThread 0] INFO org.sonarsource.scanner.maven.SonarQubeMojo - Server id: 8291CD48-AWDYQeKBkV5Xg2PFFEAk
00:07:05.728  [BuilderThread 0] INFO org.sonarsource.scanner.maven.SonarQubeMojo - User cache: /home/jenkins/.sonar/cache
00:07:05.731  [BuilderThread 0] INFO org.sonarsource.scanner.maven.SonarQubeMojo - Load/download plugins
00:07:05.731  [BuilderThread 0] INFO org.sonarsource.scanner.maven.SonarQubeMojo - Load plugins index
00:07:05.790  [BuilderThread 0] INFO org.sonarsource.scanner.maven.SonarQubeMojo - Load plugins index (done) | time=59ms
00:07:06.222  [BuilderThread 1] INFO org.apache.maven.cli.event.ExecutionEventLogger - 
00:07:06.223  [BuilderThread 1] INFO org.apache.maven.cli.event.ExecutionEventLogger - ----------------< com.my.project:my-sub-module >-----------------
00:07:06.223  [BuilderThread 1] INFO org.apache.maven.cli.event.ExecutionEventLogger - Building My Library 0.1.0-SNAPSHOT       [38/65]
00:07:06.223  [BuilderThread 1] INFO org.apache.maven.cli.event.ExecutionEventLogger - --------------------------------[ jar ]---------------------------------
6

The rabbit hole gets deeper!
Digging in a bit more and using the --fail-at-end option in maven, I can see that my build is actually successful (the failure, in fact, is because sonar starts too early and is trying to analyze modules that haven’t been built yet!).
Digging into those logs I see a lot of these warnings that don’t show up when I am running a build without sonar:

The following dependencies could not be resolved at this point of the build but seem to be part of the reactor:
<list of modules, 18 in all>
Try running the build up to the lifecycle phase "package"

So I wonder if there is a problem with how Sonar is instrumenting the build, and at what point in the flow it is actually doing so, even though it shouldn’t be until after the package phase is done. So this brings me back to my top level question: is there some other way that I can enforce that the sonar plugin doesn’t actually start install/execution until the verify phase actually starts that is different than what I am doing in the pom snippet? Do I need to add it in more sub POMs than just the top level entry point POM?

8

Hi Ben,

Multithreaded execution in maven used to be very buggy (not share what’s the current situation), and for that reason we never made the effort to make the plugin explicitly thread safe. I don’t know what version of maven you’re using, but i recommend you get the latest one hoping that it’s less problematic.

The plugin has little or no control over the orchestration done by maven. A thread safe plugin simply means that it can be executed in multiple modules in parallel without problems. The sonar plugin only runs once for the parent module, so it shouldn’t be much of a problem.

A couple questions.
Have you tried to just execute mvn -T 1C -B verify sonar:sonar and see if you get the same outcome?
Is your build info changed during the executing of other phases in a way that you need to run sonar in the same command? Otherwise an alternative is to build with multithreading and then just run mvn sonar:sonar in a separate command.

9

Hey Duarte! I was just about to come here with that exact update! It worked when running mvn -T 1C -B verify sonar:sonar! Apparently maven behaves differently when running sonar as part of a profile in a stage, but running it as a standalone goal AFTER verify works perfectly, however you MUST be on the latest version of maven for this to work (3.6.3). This failed on earlier versions of maven due to known issues there, which then also caused me to spin my wheels for a few days because I thought I had already tried this :slight_smile:.

Thank you so much for the help everyone and I hope that this thread helps others who may experience this interesting interaction with sonar and maven in the future!

1 Like
10

I’m glad it’s working, thanks for the update.

11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.