Sensor javabugs [dbd] takes 5 hours

SonarCloud
1

Since start of October sensor javabugs [dbd] takes several hours. It looks like a memory leak…
Before October whole Sonar analysis took 20 minutes. No change on our side but SonarCloud.io has contained some new improvements since start of October.

  • ALM used: GitHub
  • CI system used - internal GoCD agent, allocated 20 GB of memory
  • Scanner command used when applicable: gradlew sonar…
  • Languages of the repository: Java, JS, etc.
  • Error observed
Reading IR files from: */build/sonar/ir/java
Analyzing 94518 functions to detect bugs.

Before the hang up:

java.io.IOException: The cache stream is too big (>4772) for file *.ucfgs but the exception does not take a time...
3

Hi @msobeslavsky ,

Thank you for reporting this problem and for having already identified what sensor is slowing down your analysis.

Could you please share with us the content of /godata/pipelines/eps-core-verification/eps-core/build/sonar/ir/java ?

I will contact you privately, so you will be able to send me those files privately.

5

I can confirm same issue with “Sensor javabugs [dbd]” – it takes 6+ hours to execute and I just stop it before end. Can be “fixed” (skip) by -Dsonar.internal.analysis.dbd=false command line.

It takes 5 Gb memory by heap (after GC), here is a cpu sampling with “freeze” (after few hours of sensor javabugs works):

shot_231029_212123
shot_231029_2121231464×655 80.1 KB

Last logs on freeze:

[INFO] Sensor javabugs [dbd]
[INFO] Reading IR files from: D:\mage_last\target\sonar\ir\java
[INFO] Analyzing 189151 functions to detect bugs.

I used it for big open source project (1.5m loc): https://github.com/magefree/mage/ with all security rules disabled in custom profile (last time I used sonar with default profile – it takes 15+ hours for analyse, so disabled it to speed up).

Here is a sonar project page with latest analyse (without security and dbd parts it takes ~6 hours to complete).

How to reproduce:
1.Clone project;
2. Run build: mvn clean install -DskipTests
3. Run sonar: mvn -e sonar:sonar -Dsonar.projectKey=magefree_mage -Dsonar.organization=magefree -Dsonar.host.url=https://sonarcloud.io -Dsonar.token=xxx

1 Like
8

Hello Oleg,

Thanks a lot for your report. It allowed us to identify a few easy-to-fix performance problems when we profiled one of our analyzers (DBD) running on mage. We have applied and released the fix which has now made its way to SonarCloud. It will also be included in the next SonarQube release.

I tested the fix by running the DBD analysis on the first 50000 functions in mage and got a ~2.5x speedup. That’s significant, but the analysis will still take at least one hour, so you might want to leave it disabled for now.

The profiling I did on mage revealed some other improvements that we should make, but that are a bit more involved. We have made mid-term plans to investigate what can be done about those. I’ll keep you updated.

Cheers,

Sylvain

PS @msobeslavsky: the problem seems to be different in your case, I’ll update you privately.

1 Like