I recently upgraded our SonarQube Community Edition from version 8.0 to version 8.9.7. No hardware has changed - it still runs on the same server and connects to the same MS SQL Server database.
However, the scan times for one of our apps have increase from ~10 minutes to ~25 minutes. The app in question has ~500k lines of Java code, and uses version 3.1.1 of the Gradle scanner plugin. Upgrading to 3.3 of the Gradle plugin did not change the scan time. The scans run on our Bamboo server, and no changes have been made to it.
I tried turning on verbose logging for the scan and couldn’t see any particularly slow operations. Files seem to be getting analysed the whole time:
2022-02-17T15:41:48.712+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 57/6136 files analyzed, current file: src/main/java/aaa.java
2022-02-17T15:41:58.727+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 149/6136 files analyzed, current file: src/main/java/bbb.java
2022-02-17T15:42:08.738+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 241/6136 files analyzed, current file: src/main/java/ccc.java
2022-02-17T15:42:18.731+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 330/6136 files analyzed, current file: src/main/java/ddd.java
2022-02-17T15:42:28.750+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 425/6136 files analyzed, current file: src/main/java/eee.java
...
2022-02-17T15:51:29.236+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 6073/6136 files analyzed, current file: src/main/java/zzz.java
2022-02-17T15:51:44.853+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 51/4461 files analyzed, current file: src/test/java/aaaTest.java
...
2022-02-17T16:05:15.428+1100 [INFO] [org.sonarqube.gradle.SonarQubeTask] 4421/4461 files analyzed, current file: src/test/java/zzzTest.java
Is this increase in scan time expected? If not, how do I go about diagnosing the problem?
In fact, this performance impact is not unexpected.
Between 8.0 and 8.9, we completely rewrote Java analysis for better accuracy, which did entail a performance trade-off. In addition, we added a lot of rules, most of which are on by default in Sonar way.
Thanks for the information - that certainly explains the increase in scan time.
Was the rewrite of the Java analysis mentioned in the release notes? I read the Release Upgrade Notes for 8.9 before upgrading, but I couldn’t find mention of it there. Is there other documentation I should consult before upgrading in the future?
That’s an excellent question. You found & read what exists. Unfortunately, compiling the upgrade notes is a manual process for each release, and compiling the aggregation you read was also a manual process. So it’s quite possible for things to slip through the net & I think that’s what happened here.
I’ll ping internally to see if we want to retroactively add something.
For ~500k lines of Java code, the expected analysis duration with SQ 8.9.x CE should be around 13 minutes.
Here are some generic recommendations to remove the obvious bottlenecks:
make sure you run the scan on a Linux machine: you can get up to 50% improv. compared to a Windows box
if you have no choice and must use a Windows build agent, deactivate Windows Defender on that machine while running the scan
run the scan with JDK11+ (better perf compared to JDK8)
use SSD to get the fastest IO
One thing to try is to scan a reference project on your instance and see how long it takes on your side to scan it.
On our side, this project is scanned in 10 minutes with SQ 8.9.3 CE LTS, 8GB allocated to the scan. SSD, Ubuntu, Intel® Core™ i5-3570 CPU @ 3.40GHz × 4
Ideally, you should think about upgrading to the very latest version of SonarQube to get the best performance. Can you try the 9.3 and see what you get? (don’t compare with the first analysis which is always slower than the following analysis because we gather initial SCM data during the first scan).
Thanks for the reference project for comparison. I ran the scan on the nuxeo repository as per your instructions and it completed in 12 minutes. This was using SonarQube 8.9.7 CE hosted in AWS. The scanning machine runs Windows 10 with an i7-6700 4 core CPU @ 3.4GHz and an SSD.
The nuxeo repository has 419k LOC and is broken into many small modules. Our app has 501k LOC and is a single codebase. Could that explain why our app took twice as long (25min) to scan as nuxeo (12min)?