OOM from the Sonar Scanner after tuesday Azure DevOps Extension update

Hi,

one of our project (nodejs + cloudformation) scan pipeline is running in systematic OOM of the Sonar Scanner after the last Azure DevOps extension update (1.36.0).

I tried to set the SONAR_SCANNER_OPTS to -Xmx2048M with no benefits.

From the pipeline logs I noticed that, running the pipeline from the same commit-id, after the update there’s some difference.

BEFORE the update

AFTER the update

And here the first lines of the OOM exception

I can provide the full log in a private conversation.

Right after this last piece of logs, the scanner breaks in OOM. The above runs are from the same git commit-id, so, no code changes are involved.

I can notice that much more files are being scanned from a new “TextAndSecretSensor”.

Any clue?

BR,
Andrea

1 Like

Hello @Andrea_Migliaccio,

Sorry for the late response. Thank you for your report. The files we have in the logs are. What size are these files and are they automatically generated or part of the project?

We have adapted the text analyzer to also detect secrets. This means that more files are now included in the analysis. In your particular case, we need to see if these files are actually relevant for the analysis. If they are not, you can exclude them from the analysis. And we can try to identify a pattern to exclude these files from the scope.

Best,
Nils

Hi Nils,

I’m not completely sure to understand what you actually mean. The log files I’ve shown are generated by the CI/CD pipelines we are using to build our applications and where the SonarCloud scan is performed. They are not part of the git repo being scanned, they are just the logs of the sonar-cloud scanner while running…

If you contact me in a private conversation I can provide them and even run them in debug mode.

In the meantime we are experiencing more and more OOM error in other pipelines. The last one is a pretty big typescript / react / react-native repository what is now basically impossible to scan anymore…

Is there a way to make the scanner behave in the “old” way and hopefully avoid the OOM?

We had similar issues in the past and we were able to disable some new feature causing similar issues (sonar.internal.analysis.dbd=false).

Hope to hear from you soon,
Andrea

Hi @Andrea_Migliaccio,

thanks for the heads up. We are continuing to investigate this issue and will try to find a solution as soon as possible.

Best,
Nils

Hi @Andrea_Migliaccio,

can you share a reproducer or heap dump of the OOM? This will allow us to investigate your problem much more effectively.

In the meantime, you could exclude the directory causing the OOM from the analysis to unblock your pipeline.

Best,
Nils

Hi @Andrea_Migliaccio,

we decided to change the predicate to select files for the secret analysis. After this fix it should analyze the same files as before the update. We plan to release this fix soon. I will keep you up-to-date.

Best,
Nils

Hi @Nils_Werner ,

thanks for you effort!

I’ll keep an eye on any update to the Azure DevOps tasks to test them against our problematic repos.

Kind regards
Andrea

Hi @Andrea_Migliaccio,

the fix is released and deployed on SonarCloud. Can you confirm, that the OOM problem is solved?

Best,
Nils