Mvn multimodule build, create single SQ projects

SonarQube
1

Hi,

unfortunately we’re not running with Artifactory (yet), where Sonarqube integration seems much better
according to https://blog.sonarsource.com/fully-automated-promotion-pipelines-with-sonarqube-and-artifactory

For deployments from Sonatype Nexus, i had to create a check whether an artifactversion
is Sonarqube approved, means artifactversion = sonar.projectVersion and QualityGate = OK.
With web api, e.g. https://somesonarhost/api/project_analyses/search?project=com.foo.bar:foobar&ps=500 wrapped in a shared library with parameters projectkey and version it works fine for ‘normal’ Jenkins Maven pipelines, but not for multimodule builds.

The mvn deploy goal publishes one artifact for every modul, but the mvn sonar:sonar goal creates
a project for the parent project, so if the check runs with projectkey = groupid:artifactid and
version = artifactversion it results in 404 not found, which will prevent the deployment.

So, how do get i get multiple Sonarqube projects from a maven multi module build ?
I need one Sonarqube project for every module.

Regards,
Gilbert

2

Hi Gilbert,

You’re not going to be able to do this with a single command, but you should be able to cd into each module and analyze individually.

 
Ann

3

Hi Ann,

using 7.8 Enterprise, will update to 7.9.1 as soon as the MSSQL related bug is fixed.
It was possible to use sonar.modules = module1, module2, module3 ... in former versions of Sonarqube. Didn’t find related entries in the most recent docs and also searched Sonarsource Jira for related tickets. Reading https://jira.sonarsource.com/browse/MMF-365 it’s not clear to me if sonar.modules is still supported ?!

Gilbert

4

Hi Gilbert,

What sonar.modules did was delineate the modules inside a project for non-Maven analysis. What you want is kinda the opposite; each module as its own project outside the main project. TBH, sonar.modules probably still does its old job, since modules were only dropped on the UI side but they’re still very much a fact of life on the analysis side. But it doesn’t do what you want.

 
HTH,
Ann

5

Hi Ann,

i tried as you proposed, “cd’ing” into each module.

the Maven project structure, with 2 modules foo-broker-xxx:

<DIR>          .
<DIR>          ..
<DIR>          .git
            59 .gitignore
<DIR>          container-config
         2.863 Jenkinsfile
<DIR>          foobar-broker-common
<DIR>          foobar-broker-jee
         9.346 pom.xml

the relevant Jenkins pipeline parts:

stage('\u2778 Sonarqube Analysis') {

  def pom = readMavenPom file: 'pom.xml'
  def isMultiModuleBuild = pom.modules.size() != 0
  
  if(isMultiModuleBuild) {
    println "Maven Multimodule Build, Modules: ${pom.modules}"
    pom.modules.each {
      dir("$it") {
        println "\u27B9 Multimodule Build, start Sonarqube Analysis for Module $it.."
        withSonarQubeEnv('SONARTS') {
          withMaven(jdk: 'SUN JDK 1.8.0_171', maven: 'Maven 3.6.1', globalMavenSettingsConfig: 'nexus-read-win') {
            genericSh("mvn sonar:sonar -Dsonar.projectBaseDir=$it")
          }
        }
          println "Sonarqube Analysis for Module $it \u2714"
      }
      
      stage('\u2779 Sonarqube Quality Gate') {
        println '\u27B9 Multimodule Build, start Sonarqube Quality Gate for Module $it..'
        timeout(time:10 , unit: 'MINUTES') {
          def qg = waitForQualityGate()
          if (qg.status != 'OK') {
            error "Pipeline aborted due to quality gate failure: ${qg.status}"
          }
        }
        println "Sonarqube Quality Gate for Module $it \u2714"
      }
    }
  } else {
      println '\u27B9 start Sonarqube Analysis ..'
      withSonarQubeEnv('SONARTS') {
        withMaven(jdk: 'SUN JDK 1.8.0_171', maven: 'Maven 3.6.1', globalMavenSettingsConfig: 'nexus-read-win') {
          genericSh("mvn sonar:sonar")
        }
      }       
  }
  println 'Sonarqube Analysis \u2714 '
}

the relevant parts of the Jenkins pipeline log:

[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (❸ Sonarqube Analysis)
[Pipeline] readMavenPom
[Pipeline] echo
Maven Multimodule Build, Modules: [foobar-broker-jee, foobar-broker-common]
[Pipeline] dir
Running in E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee
[Pipeline] {
[Pipeline] echo
➹ Multimodule Build, start Sonarqube Analysis for Module foobar-broker-jee..
[Pipeline] withSonarQubeEnv
Injecting SonarQube environment variables using the configuration: SONARTS
[Pipeline] {
[Pipeline] withMaven
[withMaven] Options: []
[withMaven] Available options: 
[withMaven] using JDK installation SUN JDK 1.8.0_171
[withMaven] using Maven global settings.xml 'nexus-read-win' with Maven servers credentials provided by Jenkins (replaceAll: true): [mavenServerId: 'nexus', jenkinsCredentials: 'ae9555f8-cdb5-4a3a-b5f6-59e4d4e9325a', username: '0npLvOKs', type: 'UsernamePasswordCredentialsImpl']
[withMaven] using Maven installation 'Maven 3.6.1'
[Pipeline] {
[Pipeline] echo
mvn sonar:sonar -Dsonar.projectBaseDir=foobar-broker-jee
[Pipeline] bat

E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee>mvn sonar:sonar -Dsonar.projectBaseDir=foobar-broker-jee 
----- withMaven Wrapper script -----
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T21:00:29+02:00)
Maven home: E:\Jenkins\tools\hudson.tasks.Maven_MavenInstallation\Maven_3.6.1\apache-maven-3.6.1\bin\..
Java version: 1.8.0_171, vendor: Oracle Corporation, runtime: E:\Software\java_64\JDK\1.8.0_171\jre
Default locale: de_DE, platform encoding: Cp1252
OS name: "windows server 2012", version: "6.2", arch: "amd64", family: "windows"
[INFO] [jenkins-event-spy] Generate E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee@tmp\withMaven29f1e18e\maven-spy-20190710-154352-8118225438414407873460.log.tmp ...
[INFO] Scanning for projects...
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no dependency information available
[WARNING] Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in https://nexus.com.foo/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced
[INFO] 
[INFO] ------------------< com.foo.bar.foobar:foobar-broker-jee >-------------------
[INFO] Building foobar-broker-jee 1.1.1-SNAPSHOT
[INFO] --------------------------------[ war ]---------------------------------
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no dependency information available
[WARNING] Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in https://nexus.com.foo/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced
[INFO] 
[INFO] --- sonar-maven-plugin:3.6.0.1398:sonar (default-cli) @ foobar-broker-jee ---
[INFO] User cache: C:\Users\fredfart\.sonar\cache
[INFO] SonarQube version: 7.8.0
[INFO] Default locale: "de_DE", source code encoding: "UTF-8"
[WARNING] SonarQube scanners will require Java 11+ starting on next version
[INFO] Load global settings
[INFO] Load global settings (done) | time=919ms
[INFO] Server id: xxxxxxx-xxxxxxxx
[INFO] User cache: C:\Users\fredfart\.sonar\cache
[INFO] Load/download plugins
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=289ms
[INFO] Load/download plugins (done) | time=384ms
[WARNING] API compatibility mode is no longer supported. In case of error, plugin JavaMelody [javamelody] should package its dependencies.
[INFO] Loaded core extensions: developer-scanner
[INFO] Process project properties
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.104 s
[INFO] Finished at: 2019-07-10T15:43:57+02:00
[INFO] ------------------------------------------------------------------------
[INFO] [jenkins-event-spy] Generated E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee@tmp\withMaven29f1e18e\maven-spy-20190710-154352-8118225438414407873460.log
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project foobar-broker-jee: The base directory of the module 'name-broker-common' does not exist: E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee\name-broker-common -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Picked up JAVA_TOOL_OPTIONS: -Dmaven.ext.class.path="E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee@tmp\withMaven29f1e18e\pipeline-maven-spy.jar" -Dorg.jenkinsci.plugins.pipeline.maven.reportsFolder="E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee@tmp\withMaven29f1e18e" 
[Pipeline] }

It seems, the pipeline goes into module dir via dir("$it"), i don’t understand the error:

The base directory of the module 'name-broker-common' does not exist: E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee\name-broker-common -> [Help 1]
[ERROR]

as the folder definitively exists !?
The other way around, means taking groupid:artifactid from parent for the check is no option, as the build and deployment pipeline jobs are not connected as up/downstream jobs and the check has to be part of the deployment pipeline - so no pom.xml to get the parent for which a Sonarqube project exists. We have only the GAV parameters of the artifact (which is equal to 1 module) as build parameter, so i need a way to make the Sonar scanner create a Sonarqube project for every module of this Maven multimodule build.

Gilbert

6

Hi Gilbert,

I think this is your problem: -Dsonar.projectBaseDir=$it. This is not a parameter you should / need to use under normal circumstances. And other than wanting to analyze modules as individual projects, your qualify as “normal”.

 
HTH,
Ann

7

Hi Ann,

i first tried without -Dsonar.projectBaseDir=$it but i got the error that projectBaseDir is missing.

Gilbert

8

Hi Gilbert,

I’m kinda stabbing in the dark here. My test before I gave you my recommendation was to build a multi-module project, literally cd into a module and analyze. It worked like a charm. That’s why I suspect there’s something going on here with parameters or maybe syntax. I have two thoughts…

 

The base directory of the module ‘name-broker-common’ does not exist: E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee\name-broker-common → [Help 1]

What strikes me here is the repetition of ‘name-broker-common’. Is there a ‘name-broker-common’ inside ‘E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee\name-broker-common’?

I.e. does this exist: E:\Jenkins\workspace\OTHER\Pipelines\foobar-broker\foobar-broker-jee\name-broker-common\name-broker-common?

 
The other thought is that you’re analyzing on Windows, and \ versus / has always been a little… “fun”. So I wonder if this is going sideways at the interpolation of $it when you pass it in as sonar.projectBaseDir.

And building on that, the fact that you got an error about projectBaseDir being missing is odd, because it’s always assumed to be .. I’m not familiar with the dir command you used. Google/StackOverflow tell me that it’s an equivalent of cd, but the fact that it is a different command makes me wonder just how equivalent it is.

 
Ann

9

Hi Ann,

finally it works as expected without using -Dsonar.projectBaseDir
The name-broker-common came from a previous test with sonar.modules in the pom properties section, missed that.
After deleting this entry all fine now :slight_smile:
Thanks for taking your precious time!

Gilbert

1 Like