Java out of heap space on scanner-- unable to increase

SonarQube
,
1

Hi,

I have the following configuration/issue:

  • SonarQube Developer 9.9 LTS
  • Deployed with ZIP file on RHEL7
  • Large project running out of heap space:
##[error]Exception in thread "HttpClient-1-SelectorManager" java.lang.OutOfMemoryError: Java heap space
Exception in thread "HttpClient-1-SelectorManager" java.lang.OutOfMemoryError: Java heap space
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 8:06.020s
INFO: Final Memory: 9M/34M
INFO: ------------------------------------------------------------------------
##[error]ERROR: Error during SonarScanner execution

All articles I’ve been able to find on how to increase the memory available to the Java scanning process have not helped. I had added the following to sonar.properties, but this appears to only affect the sonarqube server process, not the scanner itself:

sonar.ce.javaOpts=-Xmx1G -Xms1G

I’ve also tried adding the following to /etc/init.d/sonarqube:

export SONAR_SCANNER_OPTS="-Xmx1024m -Xms1024m"

These are the sonarqube processes I see running right now:

sh-4.2$ ps -ef | grep sonarqube
sonarqu+  1916  1614  0 Nov03 ?        00:18:43 /usr/lib/jvm/java-17.0.2-openjdk/jdk-17.0.2/bin/java -XX:+UseG1GC -Djava.io.tmpdir=/opt/sonarqube/temp -XX:ErrorFile=/opt/sonarqube/logs/es_hs_err_pid%p.log -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djna.tmpdir=/opt/sonarqube/temp -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j2.formatMsgNoLookups=true -Djava.locale.providers=COMPAT -Dcom.redhat.fips=false -Des.enforce.bootstrap.checks=true -Xmx512m -Xms512m -XX:MaxDirectMemorySize=256m -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/opt/sonarqube/elasticsearch -Des.path.conf=/opt/sonarqube/temp/conf/es -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=false -cp /opt/sonarqube/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch
sonarqu+  2125  1614  0 Nov03 ?        00:25:31 /usr/lib/jvm/java-17.0.2-openjdk/jdk-17.0.2/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/temp -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -Dcom.redhat.fips=false -server -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttp.nonProxyHosts=localhost|127.*|[::1] -Dhttp.auth.ntlm.domain= -DsocksProxyHost= -DsocksProxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -cp ./lib/sonar-application-9.9.2.77730.jar:/opt/sonarqube/lib/jdbc/postgresql/postgresql-42.5.1.jar org.sonar.server.app.WebServer /opt/sonarqube/temp/sq-process11309343642576874711properties
sonarqu+  2245  1614  0 Nov03 ?        00:24:32 /usr/lib/jvm/java-17.0.2-openjdk/jdk-17.0.2/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/temp -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.util=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -Dcom.redhat.fips=false -Xms2G -Xmx2G -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttp.nonProxyHosts=localhost|127.*|[::1] -Dhttp.auth.ntlm.domain= -DsocksProxyHost= -DsocksProxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= -cp ./lib/sonar-application-9.9.2.77730.jar:/opt/sonarqube/lib/jdbc/postgresql/postgresql-42.5.1.jar org.sonar.ce.app.CeServer /opt/sonarqube/temp/sq-process13169928192529021107properties
ssm-user 16298 16273  0 16:01 pts/0    00:00:00 grep sonarqube

I have seen in the past also:

java -Xms8m -Xmx32m --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -jar ../../lib/sonar-application-9.9.2.77730.jar

And I believe this is the problem process, with a max of 32MB available to it regardless of the config changes made. I notice this doesn’t have the full path to the JVM and (not sure why) JAVA_HOME is not set.

However, I don’t always see this separate (scanner?) process running when the scan kicks off. Is this a child process that runs on demand, or is it long-running? And how do you actually increase the memory available to it?

Thanks very much,
Scott

2

Hi Scott,

You need to set SONAR_SCANNER_OPTS on the build agent, not on the server.

 
HTH,
Ann

3

Thanks for responding Ann. How exactly is this done? The build agent and the server are on the same VM. What script would I put this in?

Thanks again,
Scott

4

Welcome :slight_smile:

in general Sonarqube should always run on a dedicated machine !
From the console log file snippet it seems you are using Jenkins, right ?

Use https://<yourjenkinscontroller>/manage/configure and create a new environment variable
named SONAR_SCANNER_OPTS, i.e.

2023-11-10_15h43_50

Gilbert

1 Like
5

Hi Gilbert. Thanks, but the error I’ve posted about is an Azure DevOps pipeline using the SQ ADO extension; Jenkins is uninvolved (although we do scan other projects which don’t have this issue through Jenkins). To be clear, SonarQube does have its own machine, it’s just that we don’t have separate scanner nodes. I misunderstood “build agent” in Ann’s post. So, I think the point you’re both making is that I need to set this environment variable in our ADO pipeline or, better, globally in ADO?

6

Exactly, the Azure DevOps integration should be similar to Jenkins.
While it is possible to set an environment variable in your pipeline - in Jenkins it’s env.FOO = bar - of course it makes more sense to set it globally.