SonarJS: OutOfMemoryError

pkamath · August 23, 2018, 5:11pm

I tried the above command in the “begin” step. I ran into a memory issue

WARNING: WARN: [CHECKPOINT_WRITER] WARNING Transaction neither committed nor rolled back Transaction_100000002 depth=1 status=<ts=68,264 tc=UNCOMMITTED mvv=0>
Exception in thread "CHECKPOINT_WRITER" Exception in thread "CLEANUP_MANAGER" Exception in thread "Report about progress of Javascript analyzer" java.lang.OutOf
MemoryError: Java heap space
        at org.sonarsource.analyzer.commons.ProgressReport.run(ProgressReport.java:61)
        at java.lang.Thread.run(Thread.java:748)
java.lang.OutOfMemoryError: Java heap space
        at java.util.ArrayList.iterator(ArrayList.java:834)
        at com.persistit.TransactionIndex.checkpointAccumulatorSnapshots(TransactionIndex.java:1106)
        at com.persistit.CheckpointManager.createCheckpoint(CheckpointManager.java:247)
        at com.persistit.CheckpointManager.pollCreateCheckpoint(CheckpointManager.java:197)
        at com.persistit.CheckpointManager.runTask(CheckpointManager.java:300)
        at com.persistit.IOTaskRunnable.run(IOTaskRunnable.java:144)
        at java.lang.Thread.run(Thread.java:748)
java.lang.OutOfMemoryError: Java heap space
        at java.util.ArrayList.<init>(ArrayList.java:152)
        at com.persistit.CleanupManager.poll(CleanupManager.java:166)
        at com.persistit.CleanupManager.runTask(CleanupManager.java:88)
        at com.persistit.IOTaskRunnable.run(IOTaskRunnable.java:144)
        at java.lang.Thread.run(Thread.java:748)
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 22:50.494s
INFO: Final Memory: 15M/925M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarQube Scanner execution
ERROR: Java heap space
ERROR:
The SonarQube Scanner did not complete successfully
14:45:57.488  Creating a summary markdown file...
14:45:57.491  Post-processing failed. Exit code: 1

-----------------------------------------------------------------------------------------------------------------------

I went into sonar.properties and uncommented the following line and made the change (in bold)

sonar.ce.javaOpts=-Xmx1G -Xms128m -XX:+HeapDumpOnOutOfMemoryError

And re-ran, but that gave the error above.Even increasing the memory to 2G throws the same error.

This is a HUGE solution file with thousands of files of all sizes and we have had the same issues when using other static analysis files …

Colin · August 23, 2018, 9:29pm

Padmanabha,

While increasing the memory available for the compute engine could be necessary for a sufficiently large project, I would also try following the troubleshooting guide here to increase the amount of memory available to the scanner itself.

Colin

pkamath · August 30, 2018, 3:48pm

Hi Colin,

I have asked this question before but is there a way to restrict the analysis of a .NET sln file only to certain projects ? The reason why this analysis is failing is because we have a huge solution with close to 30 projects. Out of this, the crucial ones are 2 - 3 projects only. The rest are not so important or rather, of lesser priority for such analysis. Note that each project is within it’s own folder in the solution so, unless I am missing something, all I need top tell sonarqube is to ignore the files in the folders … or alternately, only analyse files in certain folders.

I’ve not had any problems running Sonarqube against smaller projects, so running SonarQube against sub-projects should, I am hoping, give us some output that is usable.

I know that there are ways we can do this by inserting xml config blocks into projects themselves (like this) but for a proof of concept scenario, I really don’t want to change our source.

Thanks !

xbourguignon · August 30, 2018, 4:28pm

Hi Padmanabha,

In order to restrict to a subset of projects, you should play with inclusions/exclusions defined in sonar.properties, check this documentation: https://docs.sonarqube.org/display/SONAR/Narrowing+the+Focus

Ammo · August 30, 2018, 4:42pm

You can go with /d:sonar.js.file.suffixes=.foo (like you did for css).

FYI we are actually working on the issues.

pkamath · August 30, 2018, 6:27pm

Sorry for the noobie question but I am not able to restrict the analysis to a certain directory using sonar.sources !

I am able to exclude the js and .css file analysis by specifying this in Administration > General Settings > Analysis Scope on the project level

I created a sonar.project.properties file in the project root dir and this is what it contains

sonar.projectKey=MyWebProject
sonar.projectName=My project
sonar.projectVersion=1.0
sonar.sources=MyWebProject.Web

I want the source files in the MyWebProject.Web directory to be analyzed (and only the cs files). However, when I run the project, it does ignore the css and js files but ALL directories in the root of the project are still included in the analysis.

As another tack, I have put sonar.sources=MywebProject.Web in the default SonarQube.Analysis.xml file and am re-running the whole thing again (fingers crossed) … but am I missing something here ?

And for the sake of completeness, here are the commands that I am using

Step 1 - SonarScanner.MSBuild.exe begin /k:“MyWebProject” /d:sonar.host.url=“http://localhost:9000” /d:sonar.login=“secret string”
Step 2 - msbuild.exe" “/t:Rebuild” “/p:Configuration=PROD” MyProject.sln
Step 3 - SonarScanner.MSBuild.exe end /d:sonar.login=“secret string”

Steps 1 and 2 run successfully

Additional note (see following post)

pkamath · August 30, 2018, 6:43pm

Now that I am looking at the output, even the file exclusion seems to be NOT working

Notice that it is still processing js files AND it is in a directory called Project.Core Which, obviously is not Project.Web

Why, oh why is this so complicated ? I am fully to blame maybe, but still …

pkamath · August 31, 2018, 9:41pm

Ok, this worked for me.

This not only cut down my processing time to less than a minute but also allowed me to focus on .NET part of the solution i.e. .cs files.

Putting the following into the SonarQube.Analysis.xml file had no effect.
MyProject.Web
Like I mentioned above, it went through ALL the folders nonetheless.