OOM crash (java.lang.OutOfMemoryError) during JsSecuritySensorV2 analysis

SonarQube Cloud
, ,
1

Running into an OOM crash on-and-off for the past few months. We’ve had to disable the required check on the merge queue since it has been blocking our CI pipeline, which is less than ideal.

Here is the requested info about our setup:

ALM: GitHub

CI system: GitHub Actions (GitHub-hosted runners, ubuntu-*, Azure-backed)

Scanner command:

SonarSource/sonarqube-scan-action@9598b8a83feef37de07f549027fab50ecffe6a6e

with sonar-project.properties at the repo root. Relevant JVM env var:

SONAR_SCANNER_JAVA_OPTS=-Xmx12G -Xms1024m

Languages: TypeScript (primary), with some YAML, CSS, HTML, JSON also scanned

Repo: GitHub - medplum/medplum: Medplum is a healthcare platform that helps you quickly develop high-quality compliant applications. · GitHub

Failed run example: Re-try fix allowing use of modifiers in _filter search · medplum/medplum@1fbd729 · GitHub

Error observed:

The scanner OOMs inside JsSecuritySensorV2 after approximately 20 minutes of analysis (762 files), while processing packages/server/src/index.ts (file 687/762). The JVM was configured with -Xmx12G at the time of the crash.

java.lang.OutOfMemoryError: Java heap space
	at scala.collection.immutable.List.prependedAll(List.scala:156)
	at scala.collection.IterableOnceOps.toList(IterableOnce.scala:1446)
	at scala.collection.IterableOnceOps.toList$(IterableOnce.scala:1446)
	at scala.collection.AbstractIterable.toList(Iterable.scala:935)
	at com.sonar.a.d.c.b.e.g(SourceFile:152)
	at com.sonar.a.d.c.b.e.d(SourceFile:42)
	at com.sonar.a.d.c.b.h.g(SourceFile:169)
	at com.sonar.a.d.c.at.g(SourceFile:243)
	at com.sonar.a.d.c.ap.b(SourceFile:629)
	... (repeated flatMap recursion on scala.collection.immutable.List)
	at cats.data.IndexedStateT.flatMap$$anonfun$1$$anonfun$1$$anonfun$1(IndexedStateT.scala:45)
	at cats.data.OptionT.flatMap$$anonfun$1(OptionT.scala:207)
	at cats.data.WriterT.flatMap$$anonfun$1(WriterT.scala:203)

File that triggers the OOM: packages/server/src/index.ts (file 687 of 762 in the JsSecuritySensorV2 pass). This file is the Express server entry point for a large FHIR backend and transitively imports a very large portion of the monorepo.

The analysis seems to be going smoothly until it gets to that file and then after 4 minutes of getting stuck on the specified file, the OOM error occurs.

Here are the logs for the aforementioned run with verbose mode enabled:
oom-sonarqube-cloud-logs.txt (1.6 MB)

Any help in debugging would be greatly appreciated.

5

Hi team,

We are also facing the same issue with Sonar scanning. Kindly check and advice. tried with SONAR_SCANNER_JAVA_OPTS: “-Xmx6g”. Not worked. Please check and assist on the same

04:31:41.749 INFO Sensor PythonSecuritySensor [security] (done) | time=4162ms
04:31:41.776 INFO ------------- Run sensors on project
04:31:41.941 INFO Sensor JsSecuritySensorV2 [jasmin]
04:31:41.993 INFO 63 file(s) will be analysed by SonarJasmin.
04:31:47.296 INFO Analysis progress: 9% (6/63 files)
04:31:47.300 INFO Analysis progress: 19% (12/63 files)
04:31:47.300 INFO Analysis progress: 28% (18/63 files)
04:31:47.303 INFO Analysis progress: 38% (24/63 files)
04:31:47.307 INFO Analysis progress: 47% (30/63 files)
04:31:47.311 INFO Analysis progress: 57% (36/63 files)
04:31:47.314 INFO Analysis progress: 66% (42/63 files)
##[error]04:45:03.379 ERROR [stderr] java.lang.OutOfMemoryError: Java heap space
04:45:03.379 ERROR [stderr] java.lang.OutOfMemoryError: Java heap space
##[error]04:45:03.381 ERROR [stderr] at scala.collection.immutable.BitmapIndexedSetNode.copyAndInsertValue(HashSet.scala:801)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.BitmapIndexedSetNode.updated(HashSet.scala:518)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.HashSet.incl(HashSet.scala:83)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.Set$Set4.incl(Set.scala:46)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.BitmapIndexedSetNode.copyAndInsertValue(HashSet.scala:801)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.BitmapIndexedSetNode.updated(HashSet.scala:518)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.HashSet.incl(HashSet.scala:83)
04:45:03.381 ERROR [stderr] at scala.collection.immutable.Set$Set4.incl(Set.scala:46)
##[error]04:45:03.382 ERROR [stderr] at scala.collection.immutable.Set$Set4.incl(Set.scala:295)

8

Hello @ThatOneBro, thank you for reporting this!

We have resolved the issue, and the JS/TS security analysis for medplum now completes gracefully within a reasonable timeframe.

We plan to deploy the fix to SonarQube Cloud this week. Once it’s live, we’d be happy to open a PR on your repository to revert this change :grimacing:

@Thamulansar, we hope this fix addresses your issue as well. However, it’s difficult to confirm if the two are related based solely on that snippet of the stack trace.

Could you please check if the problem persists once the new version is deployed? If it does, feel free to open a new thread and we’ll be happy to take a closer look.

Best,
Samuele

12

Thank you @Samuele_Buro, that’s great to hear. We’ve been relying on Sonar for a few years and so it’s not been great to fly without it for the last few weeks.

We would be open to that PR if you would like to throw it up after the change is live.

Thanks to you and the team for your hard work!

Best,
Derrick

15

Hi @Samuele_Buro, just wanted to call out that we’ve re-enabled Sonar for all PRs and merge groups successfully and things seem to be running smoothly.

A big thank you to you and the team for figuring it out. Our coverage went down by 0.5% in the time we had to disable Sonar so we are excited to have it back :smile:

Cheers!

Derrick