SonarQube analysis hangs in the "Running Tarjan" step or throws OutofMemoryError

SonarQube
,
1

Must-share information (formatted with Markdown):

  • SonarQube 8.1.0.31237 Enterprise

  • Ant Task Jar sonarqube-ant-task-2.7.0.1612.jar

  • what are you trying to achieve
    Full scan of the master branch of our code repository. Encountering outofmemory error or hang of the scan before the scan reaches the server side analysis.

  • what have you tried so far to achieve this

  1. Increased memory for various processes using SONAR_SCANNER_OPTS, ANT_OPTS, -Dsonar.ce.javaOpts. Even increasing to 8g doesn’t work.
  2. Excluded source files. It works when I drop the file count from 1000+ to 500 odd.
  3. I’ve also excluded some giant class files which have 10k + LoC

I’ve extracted the specific problematic section from Jenkins log here for both cases - OutofMemory and hang. Any insight would be appreciated.

Jenkins - Log - Excerpt - OutofMemoryError.txt (13.8 KB) Jenkins - Log - Excerpt - Hang.txt (7.4 KB)

2

If you’re staying on a non-LTS release of SonarQube, I’d recommend you upgrade to the latest version (v8.2 currently v8.3 next week). Each release typically contains some performance improvements for our security analysis engine, which is what is causing the issue here.

If you want to throw more memory at the scacnner, you should use the SONAR_SCANNER_OPTS environment variable and just include heap sizes (-Xmx8G -Xms8G). sonar.ce.javaOpts has no meaning here.

3

I’ll try with 8.2 and get back on this topic.

4

I tried with 8.2.0.32929, still have the same problem. Increasing memory didn’t help (I tried up to 8GB). I then tried deactivating all Java rules that were either Security Hotspot or Vulnerability categories. The analysis completed successfully.

Can you provide any suggestions on a smaller set of rules to deactivate to achieve the same result? It would also help if the issue can be narrowed down to a specific file causing the issue so that I can directly exclude that file. I’m not sure how to identify which specific file is breaking this. I’ve enabled DEBUG mode and verbose logging but in this specific step there is no file name detail being shown in the ant output.

5

Unfortunately, it’s not as trivial as excluding individual files.

Can you check that the scanner is definitely recognising the SONAR_SCANNER_OPTS environment variable? You should see something like this at the begeinning of thee scanner logs.

INFO: Scanner configuration file: /Users/colin/Tools/sonar-scanner-4.0.0.1744/conf/sonar-scanner.properties
INFO: Project root configuration file: NONE
INFO: SonarQube Scanner 4.0.0.1744
INFO: Java 11.0.3 Oracle Corporation (64-bit)
INFO: Mac OS X 10.15.4 x86_64
INFO: SONAR_SCANNER_OPTS=-Xmx2048M -Xms2048M
INFO: User cache: /Users/colin/.sonar/cache
6

I’m currently executing Sonar as an Ant task within a Jenkins pipeline. I didn’t see the SONAR_SCANNER_OPTS parameter on that log. In order to test what you asked for, I manually invoked the sonar-scanner executable with SONAR_SCANNER_OPTS set in the system environment. It actually showed up in the log and the execution completed successfully even without disabling any rules.

So, clearly the following pipeline stage is not working as intended. I will debug this but should I directly invoke the sonar scanner binary from the pipeline instead of using Ant?

stage (‘SonarQube Full Analysis’) {
environment {
SONAR_SCANNER_OPTS = “-Xmx4096m”
}
when {
anyOf {
branch ‘master’;
branch pattern: “release-*”, comparator: “REGEXP”;
}
}

		steps {
			step ([$class: 'CopyArtifact', projectName: 'DEPEND_PROJ', target: 'DEPD']);
			
			withSonarQubeEnv('sonarqube') {
				withAnt(installation: 'Ant') {
					sh "ant -v sonar -buildfile build.xml -Dsonar.ce.javaOpts='-Xmx4096m -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=128m -Xms512m -XX:+HeapDumpOnOutOfMemoryError'"
				}
			}
		}
	}