Analysis either freezes or exits with memory error after upgrading

@janes not sure how to proceed, maybe someone else can chime in @saberduck?

I’m still struck by the log that says available heap memory is super low.

Do you happened to be using Maven? If so, I would try using the recommended export MAVEN_OPTS= instead of the generic SONAR_SCANNER_OPTS=.

Not using Maven at all, and the tool did echo back the 24G setting… it’s like it echoed it, but just didn’t pass it on. My only option right now is to roll-back to the old version, but paying for access to the developer edition and not being able to use recent versions is going to be a bad mark against SQ… after it took so long to convince management that it was the best solution. Long story short, I’m quite desperate to get this addressed and avoid that kind of mess… and willing to try anything or have a screen-sharing session to provide SS with the info needed to ID the cause.

Hi @janes,

it looks like memory explodes while trying to serialize the cache of ucfg files. Ucfg files are an intermediary representation of your source code used during the analysis. Could you please check the size of ucfg2 directory which should be created under /local-path/.scannerwork directory?

I also don’t understand why the heap setting isn’t reflected in the “Final memory” log, it looks like heap is still set to 60M, which would explain the failure.

Will do a test run now… checked the build hosts to see how much space is on that disk for each of them, and:

/dev/root       497G  326G  171G  66% /
/dev/root       497G  260G  237G  53% /
/dev/root       497G  147G  350G  30% /
/dev/root       497G  200G  297G  41% /

So it seems unlikely that we’re running out of space for this. I’ll report back when I get the size of the directory that is generated in a few minutes.

So far… 167M.

Ok, we’ve been stuck at:

WARN: Using Node.js version 14 to execute analysis is deprecated and will stop being supported no earlier than May 1st, 2023. Please upgrade to a newer LTS version of Node.js [16, 18]

for over 30m, and the size is still 167M.

And finally:

WARN: Using Node.js version 14 to execute analysis is deprecated and will stop being supported no earlier than May 1st, 2023. Please upgrade to a newer LTS version of Node.js [16, 18]
[3634]INFO: Hit the cache for 0 out of 0
[3635]INFO: Miss the cache for 0 out of 0
[3636]ERROR: isAlive was interrupted
[3637]java.lang.InterruptedException: null
[3638] at java.base/java.util.concurrent.CompletableFuture.reportGet(Unknown Source)
[3639] at java.base/java.util.concurrent.CompletableFuture.get(Unknown Source)
[3640] at java.net.http/jdk.internal.net.http.HttpClientImpl.send(Unknown Source)
[3641] at java.net.http/jdk.internal.net.http.HttpClientFacade.send(Unknown Source)
[3642] at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.isAlive(EslintBridgeServerImpl.java:405)
[3643] at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.heartbeat(EslintBridgeServerImpl.java:138)
[3644] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[3645] at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
[3646] at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
[3647] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[3648] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[3649] at java.base/java.lang.Thread.run(Unknown Source)

and the directory is still just 167M.

So, our previous release branch is still passing… which means the changes to our product is prompting this issue. The changes include a new web UI that we’re working on, which makes use of several additional React packages. Would it help if I provided the list? There must be something in there that SQ is very unhappy with.

And while we’re waiting… is there any way at all to just completely turn off the JS/TS scanning to get us back up and running with C/C++ scanning?

Hi,

As a stop-gap, you could edit your sonar.sources to omit the JS files. If that’s not practical (they’re too thoroughly intermixed) then an exclusion on **/*.js should do it.

 
HTH,
Ann

Ann,

Using the exclusion did get us a passing run, thanks for suggesting that as a work-around. We obviously will need a solution for the JS side of things, since that is an important part of our SW stack. If anything is needed from me to help you guys ID the root cause, please let me know.

Brian

1 Like

@ganncamp based on the replies in this thread I have excluded some JSON/HTML files (which can be very big, 20Mb+) from our tests using the exclusions option:

sonar.test.exclusions=**/mock-detail-responses.html,**/mock-detail-responses.json,**/mock-list-response.html,**/mock-list-response.json

We use these files as input in our tests but they don’t have any analysis value. So even though this is a ‘work around’ option, it seems to work for us.

1 Like

@MichielEco, interesting. I had tried limiting the size of the JS files being analyzed via sonar.javascript.maxFileSize=1, but it had no impact on the issue here.

Hi @MichielEco,

I’m glad it solved your problem. And it’s worth noting that if the files were just input to tests, excluding them from analysis probably needed doing anyway. :smiley:

@janes I’m not aware of sonar.javascript.maxFileSize being a supported parameter.

 
Ann

@ganncamp, see Sonar scanner not including a file from my project - #12 by Andrey_Tyukin

1 Like