Sonarqube scanner cli - huge memory to find report - out of memory exception waitForQualityGate

Hi,
we are using SonarQube Dev Version 8.9 LTS and the sonar scanner cli via Jenkins.
Our setup is pretty straight forward. On a AWS EC2 we are running the sonar scanner cli.

The analysis finishes successfully but the step waitForQualityGate() fails due to a out of memory exception.

03:15:53  INFO: ANALYSIS SUCCESSFUL, you can browse http://zzz.swarovski.com/dashboard?id=sac-hybris&pullRequest=11734
03:15:53  INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
03:15:53  INFO: More about the report processing at http://zzz.swarovski.com/api/ce/task?id=AYH6Rs5VFTc1yXdzelZS
03:16:00  INFO: Analysis total time: 1:09:07.168 s
03:16:00  INFO: ------------------------------------------------------------------------
03:16:00  INFO: EXECUTION SUCCESS
03:16:00  INFO: ------------------------------------------------------------------------
03:16:00  INFO: Total time: 1:09:15.565s
03:16:00  INFO: Final Memory: 139M/2G
03:16:00  INFO: ------------------------------------------------------------------------

Jenkins Pipeline

withSonarQubeEnv(sonarQubeEnv) {
  sh "sonar-scanner -Dsonar.projectVersion=$version -Dsonar.projectKey=$SONAR_PROJECT -Dsonar.projectName=$SONAR_PROJECT"
}
sleep(10) // used to prevent lock when SonarQube does respond too fast.
timeout(time: 10, unit: 'MINUTES') {
def qaGate = waitForQualityGate() # fails


[Pipeline] End of Pipeline
java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:3181)
	at java.util.Vector.grow(Vector.java:271)
	at java.util.Vector.ensureCapacityHelper(Vector.java:251)
	at java.util.Vector.ensureCapacity(Vector.java:236)
	at org.apache.tools.ant.util.VectorSet.doAdd(VectorSet.java:86)
	at org.apache.tools.ant.util.VectorSet.addElement(VectorSet.java:98)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1245)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1194)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1156)
	at org.apache.tools.ant.DirectoryScanner.checkIncludePatterns(DirectoryScanner.java:954)
Caused: java.io.IOException: Remote call on EC2 (Frankfurt) - CommonSlave (i-08506e7ed1ccb0917) failed
	at hudson.remoting.Channel.call(Channel.java:1004)
	at hudson.FilePath.act(FilePath.java:1194)
	at hudson.FilePath.act(FilePath.java:1183)
	at hudson.FilePath.list(FilePath.java:2133)
	at hudson.FilePath.list(FilePath.java:2116)
	at hudson.FilePath.list(FilePath.java:2100)
	at hudson.plugins.sonar.utils.SonarUtils.extractReportTask(SonarUtils.java:89)
	at hudson.plugins.sonar.utils.SonarUtils.addBuildInfoTo(SonarUtils.java:118)
	at hudson.plugins.sonar.SonarBuildWrapper$AddBuildInfo.tearDown(SonarBuildWrapper.java:201)
	at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Callback.finished(CoreWrapperStep.java:192)
	at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Execution2$Callback2.finished(CoreWrapperStep.java:146)
	at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution$TailCall.lambda$onSuccess$0(GeneralNonBlockingStepExecution.java:140)
	at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution.lambda$run$0(GeneralNonBlockingStepExecution.java:77)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

The root cause is the way how the report is extracted, namely by a recursive search what consumes gigs of memory. This of course requires to use EC2 with more memory 32G instead of 8G.

The error is located in SonarUtils.java - sonar-scanner-jenkins/SonarUtils.java at master · SonarSource/sonar-scanner-jenkins · GitHub

It would be great to have a property to overwrite or to define a static location of the report-task.txt (REPORT_TASK_FILENAME). It’s usually located in .scannerwork/report-task.txt

Hey there.

Another user reported this recently.

I’ve flagged this thread for some extra attention.

Hi @GregorKoflerSwa ,

Thanks for the report. I’ve opened a ticket here to address this issue.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Hi!

It’s just a post that we are also facing this issue : Sonarqube scanner cli - huge memory to find report - out of memory exception waitForQualityGate

For the moment we didn’t find any workaround. This is a mono-repo with a quite huge workspace.

Let use know if the community can help addressing this issue (submit a PR maybe ? sonar-scanner-jenkins/SonarUtils.java at master · SonarSource/sonar-scanner-jenkins · GitHub).

Best regards,

This is the logs

[03:25:37.292Z] - Caused: java.io.IOException: Remote call on JNLP4-connect connection from 10.128.4.83/10.128.4.83:55392 failed
[03:25:37.292Z] - 	at hudson.remoting.Channel.call(Channel.java:1004)
[03:25:37.292Z] - 	at hudson.FilePath.act(FilePath.java:1192)
[03:25:37.292Z] - 	at hudson.FilePath.act(FilePath.java:1181)
[03:25:37.292Z] - 	at hudson.FilePath.list(FilePath.java:2131)
[03:25:37.292Z] - 	at hudson.FilePath.list(FilePath.java:2114)
[03:25:37.292Z] - 	at hudson.FilePath.list(FilePath.java:2098)
[03:25:37.292Z] - 	at hudson.plugins.sonar.utils.SonarUtils.extractReportTask(SonarUtils.java:91)
[03:25:37.292Z] - 	at hudson.plugins.sonar.utils.SonarUtils.addBuildInfoTo(SonarUtils.java:131)
[03:25:37.292Z] - 	at hudson.plugins.sonar.SonarBuildWrapper$AddBuildInfo.tearDown(SonarBuildWrapper.java:215)
[03:25:37.292Z] - 	at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Callback.finished(CoreWrapperStep.java:207)
[03:25:37.292Z] - 	at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Execution2$Callback2.finished(CoreWrapperStep.java:150)
1 Like

Another one occurrence java.lang.OutOfMemoryError: Java heap space in jenkins sonar scan stage - #21 by farcop

Just workarounded

stage('sonarqube-scan') { 
    withSonarQubeEnv(envOnly: true) {    
      sh "bazel run //:sq -- -Dsonar.host.url=${env.SONAR_HOST_URL} -Dsonar.login=${env.SONAR_AUTH_TOKEN} -Dsonar.projectVersion=${kcpVersion}"
    }
    
    // Workaround https://sonarsource.atlassian.net/browse/SONARJNKNS-355
    sh "mkdir -p tmp/ && cp --target-directory=tmp/ --dereference bazel-bin/sq.runfiles/kublr/.scannerwork/report-task.txt"
    dir("${env.WORKSPACE}/tmp") {
      withSonarQubeEnv() {}

      timeout(time: 15, unit: 'MINUTES') { // Just in case something goes wrong, pipeline will be killed after a timeout
        def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
        if (qg.status != 'OK') {
          error "Pipeline aborted due to quality gate failure: ${qg.status}"
        }
      }
    }
}
1 Like