Sonar Developer 10.4 slow compared to Sonar Community 9.6.1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    SonarQube 10.4.1.88267 Dev VS SonarQube 9.6.1 Community
  • how is SonarQube deployed: zip, Docker, Helm
    zip
  • what are you trying to achieve
    Move ANT project to SonarQube developer (using sonar-scanner.properties)
  • what have you tried so far to achieve this
    Run SonarQube developer in many configurations, with DB on the same machine, on another machine, and many more.

With SonarQube 9.6.1:

14:03:08  INFO: Analysis total time: 6:43.655 s
14:03:08  INFO: -------------------------------
14:03:08  INFO: EXECUTION SUCCESS
14:03:08  INFO: -------------------------------
14:03:08  INFO: Final Memory: 242M/840M

With SonarQube 10.4.1.88267:

14:33:35.016 INFO: Analysis total time: 20:28.490 s
14:33:35.020 INFO: ------------------------------------------------------------------------
14:33:35.020 INFO: EXECUTION SUCCESS
14:33:35.020 INFO: ------------------------------------------------------------------------
14:33:35.021 INFO: Total time: 20:30.552s
14:33:35.848 INFO: Final Memory: 1375M/3474M
14:33:35.848 INFO: ------------------------------------------------------------------------

When a MR is scanned, this time drops to 10~12 minutes. But main remains slow, no matter what.

Still twice the time of the old SonarQube, that is consistent between branches.

I have tried many configurations, like memory, number of cores, using another server for the database, and more.

Sonar Developer version 10

14:17:32.148 INFO: Sensor JavaSensor [java] (done) | time=247502ms 
14:18:01.200 INFO: Sensor TextAndSecretsSensor [text] (done) | time=23920ms
14:18:29.104 INFO: Sensor JavaSensor [java] (done) | time=27615ms
14:19:09.062 INFO: Sensor JavaSensor [java] (done) | time=36226ms
14:20:11.144 INFO: Sensor JavaSensor [java] (done) | time=38658ms
14:20:34.876 INFO: Sensor JavaSensor [java] (done) | time=18943ms
14:22:58.490 INFO: Sensor JavaSensor [java] (done) | time=133287ms
14:23:17.337 INFO: Sensor TextAndSecretsSensor [text] (done) | time=16951ms
14:23:54.844 INFO: Sensor JavaSensor [java] (done) | time=37434ms
14:24:30.569 INFO: Sensor JavaSensor [java] (done) | time=29981ms
14:26:15.767 INFO: Sensor JavaSensor [java] (done) | time=97738ms
14:26:29.188 INFO: Sensor TextAndSecretsSensor [text] (done) | time=11810ms
 
14:26:52.790 INFO: Sensor JavaSensor [java] (done) | time=15686ms
14:30:12.266 INFO: Sensor javabugs [dbd] (done) | time=192949ms
14:32:52.134 INFO: Sensor JavaSecuritySensor [security] (done) | time=159868ms 
 
14:33:15.480 INFO: Analysis report generated in 20396ms, dir size=93.4 MB
14:33:33.303 INFO: Analysis report compressed in 17822ms, zip size=58.4 MB

Thanks for any hint.

Hi,

Can you enable debug logging (sonar.log.level=DEBUG or sonar.verbose=true) so we can see where the time is going?

Also, SonarScanner for Ant has been deprecated. If you haven’t already moved to exec-ing SonarScanner CLI, you’ll want to do that soon.

 
Ann

Thanks for your replay.

We’re using the SonarScanner 5 CLI. But the same happens when using version 4.

Here it goes the zipped debug log:

sonar-developer.zip (3.3 MB)

Hi,

Thanks for the log. It’s a lot :joy: so I found it easiest to grep for the lines with (done) and look at the timings:

14:13:06.217 INFO: Load global settings (done) | time=120ms
14:13:06.317 INFO: Load plugins index (done) | time=76ms
14:13:06.406 INFO: Load/download plugins (done) | time=88ms
14:13:07.185 INFO: Process project properties (done) | time=53ms
14:13:07.238 INFO: Load project settings for component key: 'xxxx_xxxx-sonar_cd6051c5-5e7a-44fa-a8d8-30e6572559f9' (done) | time=17ms
14:13:07.300 INFO: Load project branches (done) | time=13ms
14:13:07.306 INFO: Load branch configuration (done) | time=5ms
14:13:07.457 INFO: Load quality profiles (done) | time=63ms
14:13:09.536 INFO: Load active rules (done) | time=2048ms
14:13:20.755 INFO: Load/download plugins (done) | time=50ms
14:13:21.014 INFO: Load project repositories (done) | time=79ms
14:13:22.682 INFO: Load metrics repository (done) | time=14ms
14:17:32.148 INFO: Sensor JavaSensor [java] (done) | time=247502ms
14:17:32.153 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=5ms
14:17:32.155 INFO: Sensor SurefireSensor [java] (done) | time=2ms
14:17:34.200 INFO: Sensor XML Sensor [xml] (done) | time=2045ms
14:17:37.280 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3080ms
14:18:01.200 INFO: Sensor TextAndSecretsSensor [text] (done) | time=23920ms
14:18:01.347 INFO: Sensor IaC Docker Sensor [iac] (done) | time=146ms
14:18:01.357 INFO: Sensor Serverless configuration file sensor [security] (done) | time=10ms
14:18:01.362 INFO: Sensor AWS SAM template file sensor [security] (done) | time=5ms
14:18:01.366 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=4ms
14:18:29.104 INFO: Sensor JavaSensor [java] (done) | time=27615ms
14:18:29.104 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
14:18:29.104 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:18:29.828 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=724ms
14:18:32.738 INFO: Sensor TextAndSecretsSensor [text] (done) | time=2910ms
14:18:32.755 INFO: Sensor IaC Docker Sensor [iac] (done) | time=17ms
14:18:32.756 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:18:32.756 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:18:32.756 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:19:09.062 INFO: Sensor JavaSensor [java] (done) | time=36226ms
14:19:09.063 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:19:09.063 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:19:09.909 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=846ms
14:19:13.968 INFO: Sensor TextAndSecretsSensor [text] (done) | time=4059ms
14:19:13.982 INFO: Sensor IaC Docker Sensor [iac] (done) | time=14ms
14:19:13.983 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:19:13.983 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:19:13.983 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:19:15.771 INFO: Sensor JavaSensor [java] (done) | time=1710ms
14:19:15.771 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
14:19:15.771 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:19:16.159 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=388ms
14:19:16.465 INFO: Sensor TextAndSecretsSensor [text] (done) | time=305ms
14:19:16.478 INFO: Sensor IaC Docker Sensor [iac] (done) | time=13ms
14:19:16.478 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:19:16.478 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:19:16.479 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=1ms
14:19:24.635 INFO: Sensor JavaSensor [java] (done) | time=8085ms
14:19:24.635 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
14:19:24.635 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:19:25.005 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=370ms
14:19:26.179 INFO: Sensor TextAndSecretsSensor [text] (done) | time=1174ms
14:19:26.189 INFO: Sensor IaC Docker Sensor [iac] (done) | time=10ms
14:19:26.190 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:19:26.190 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:19:26.190 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:19:30.965 INFO: Sensor JavaSensor [java] (done) | time=4712ms
14:19:30.966 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:19:30.966 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:19:31.400 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=434ms
14:19:32.395 INFO: Sensor TextAndSecretsSensor [text] (done) | time=994ms
14:19:32.402 INFO: Sensor IaC Docker Sensor [iac] (done) | time=7ms
14:19:32.402 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:19:32.402 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:19:32.402 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:20:11.144 INFO: Sensor JavaSensor [java] (done) | time=38658ms
14:20:11.146 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:20:11.146 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:20:12.007 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=861ms
14:20:15.846 INFO: Sensor TextAndSecretsSensor [text] (done) | time=3839ms
14:20:15.859 INFO: Sensor IaC Docker Sensor [iac] (done) | time=13ms
14:20:15.860 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:20:15.861 INFO: Sensor AWS SAM template file sensor [security] (done) | time=1ms
14:20:15.861 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:20:34.876 INFO: Sensor JavaSensor [java] (done) | time=18943ms
14:20:34.876 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
14:20:34.876 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:20:35.420 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=544ms
14:20:37.298 INFO: Sensor TextAndSecretsSensor [text] (done) | time=1878ms
14:20:37.304 INFO: Sensor IaC Docker Sensor [iac] (done) | time=6ms
14:20:37.304 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:20:37.304 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:20:37.305 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=1ms
14:20:43.875 INFO: Sensor JavaSensor [java] (done) | time=6531ms
14:20:43.875 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
14:20:43.875 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:20:44.315 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=440ms
14:20:45.125 INFO: Sensor TextAndSecretsSensor [text] (done) | time=810ms
14:20:45.133 INFO: Sensor IaC Docker Sensor [iac] (done) | time=7ms
14:20:45.133 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:20:45.133 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:20:45.133 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:22:58.490 INFO: Sensor JavaSensor [java] (done) | time=133287ms
14:22:58.494 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=3ms
14:22:58.494 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:23:00.386 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=1891ms
14:23:17.337 INFO: Sensor TextAndSecretsSensor [text] (done) | time=16951ms
14:23:17.347 INFO: Sensor IaC Docker Sensor [iac] (done) | time=10ms
14:23:17.348 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:23:17.349 INFO: Sensor AWS SAM template file sensor [security] (done) | time=1ms
14:23:17.350 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=1ms
14:23:54.844 INFO: Sensor JavaSensor [java] (done) | time=37434ms
14:23:54.845 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:23:54.845 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:23:54.920 INFO: Sensor XML Sensor [xml] (done) | time=75ms
14:23:55.752 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=832ms
14:24:00.519 INFO: Sensor TextAndSecretsSensor [text] (done) | time=4767ms
14:24:00.534 INFO: Sensor IaC Docker Sensor [iac] (done) | time=15ms
14:24:00.535 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:24:00.535 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:24:00.536 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=1ms
14:24:30.569 INFO: Sensor JavaSensor [java] (done) | time=29981ms
14:24:30.570 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:24:30.570 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:24:31.326 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=756ms
14:24:34.657 INFO: Sensor TextAndSecretsSensor [text] (done) | time=3330ms
14:24:34.665 INFO: Sensor IaC Docker Sensor [iac] (done) | time=7ms
14:24:34.666 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:24:34.666 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:24:34.666 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:24:37.207 INFO: Sensor JavaSensor [java] (done) | time=2479ms
14:24:37.208 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:24:37.208 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:24:37.613 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=405ms
14:24:37.974 INFO: Sensor TextAndSecretsSensor [text] (done) | time=361ms
14:24:37.980 INFO: Sensor IaC Docker Sensor [iac] (done) | time=6ms
14:24:37.980 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:24:37.980 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:24:37.980 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:26:15.767 INFO: Sensor JavaSensor [java] (done) | time=97738ms
14:26:15.769 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=2ms
14:26:15.769 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:26:17.378 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=1609ms
14:26:29.188 INFO: Sensor TextAndSecretsSensor [text] (done) | time=11810ms
14:26:29.199 INFO: Sensor IaC Docker Sensor [iac] (done) | time=11ms
14:26:29.201 INFO: Sensor Serverless configuration file sensor [security] (done) | time=2ms
14:26:29.202 INFO: Sensor AWS SAM template file sensor [security] (done) | time=1ms
14:26:29.204 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=1ms
14:26:33.165 INFO: Sensor JavaSensor [java] (done) | time=3910ms
14:26:33.166 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:26:33.166 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:26:33.547 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=381ms
14:26:33.898 INFO: Sensor TextAndSecretsSensor [text] (done) | time=351ms
14:26:33.919 INFO: Sensor IaC Docker Sensor [iac] (done) | time=20ms
14:26:33.920 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:26:33.920 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:26:33.920 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:26:36.297 INFO: Sensor JavaSensor [java] (done) | time=2129ms
14:26:36.298 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:26:36.298 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:26:36.722 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=424ms
14:26:37.035 INFO: Sensor TextAndSecretsSensor [text] (done) | time=312ms
14:26:37.046 INFO: Sensor IaC Docker Sensor [iac] (done) | time=11ms
14:26:37.046 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:26:37.046 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:26:37.046 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:26:52.790 INFO: Sensor JavaSensor [java] (done) | time=15686ms
14:26:52.791 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:26:52.791 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:26:53.317 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=526ms
14:26:55.658 INFO: Sensor TextAndSecretsSensor [text] (done) | time=2341ms
14:26:55.667 INFO: Sensor IaC Docker Sensor [iac] (done) | time=8ms
14:26:55.667 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:26:55.667 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:26:55.667 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:26:58.111 INFO: Sensor JavaSensor [java] (done) | time=2391ms
14:26:58.111 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=0ms
14:26:58.111 INFO: Sensor SurefireSensor [java] (done) | time=0ms
14:26:58.544 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=433ms
14:26:58.841 INFO: Sensor TextAndSecretsSensor [text] (done) | time=297ms
14:26:58.847 INFO: Sensor IaC Docker Sensor [iac] (done) | time=5ms
14:26:58.847 INFO: Sensor Serverless configuration file sensor [security] (done) | time=0ms
14:26:58.847 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:26:58.847 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:26:58.894 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=1ms
14:26:59.290 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=396ms
14:26:59.309 INFO: Sensor TextAndSecretsSensor [text] (done) | time=19ms
14:26:59.316 INFO: Sensor IaC Docker Sensor [iac] (done) | time=7ms
14:26:59.317 INFO: Sensor Serverless configuration file sensor [security] (done) | time=1ms
14:26:59.317 INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
14:26:59.317 INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
14:30:12.266 INFO: Sensor javabugs [dbd] (done) | time=192949ms
14:30:12.266 INFO: Sensor pythonbugs [dbd] (done) | time=0ms
14:32:52.134 INFO: Sensor JavaSecuritySensor [security] (done) | time=159868ms
14:32:52.136 INFO: Sensor CSharpSecuritySensor [security] (done) | time=1ms
14:32:52.137 INFO: Sensor PhpSecuritySensor [security] (done) | time=1ms
14:32:52.138 INFO: Sensor PythonSecuritySensor [security] (done) | time=1ms
14:32:52.139 INFO: Sensor JsSecuritySensor [security] (done) | time=1ms
14:32:52.500 INFO: Sensor Zero Coverage Sensor (done) | time=321ms
14:32:52.503 INFO: Sensor Java CPD Block Indexer (done) | time=3ms
14:32:52.509 INFO: CPD Executor CPD calculation finished (done) | time=0ms

In doing that, nothing stands out to me as being egregious.

Of the four longest steps,

14:17:32.148 INFO: Sensor JavaSensor [java] (done) | time=247502ms // 4.125min
14:22:58.490 INFO: Sensor JavaSensor [java] (done) | time=133287ms // 2.221min
14:30:12.266 INFO: Sensor javabugs [dbd] (done) | time=192949ms // 3.215 min
14:32:52.134 INFO: Sensor JavaSecuritySensor [security] (done) | time=159868ms // 2.664 min

the longest is just over 4min. So I think what we’re looking at here is the result of the improvement over time of the analysis - which unfortunately, but naturally takes more time - in a very large project.

It’s possible that giving analysis more memory would speed it up a little. I looked up how to do that for Maven and for Gradle before I double-checked the analysis command in your log, which seems to be… the vanilla Scanner CLI?

Switching to the scanner that matches how you build would ease the configuration burden for you (not much of a burden, I know, since analysis is already up and running). It’s possible that it would also give a more precise list of dependencies to analysis, which might shave a fraction of time off of analysis but I really kinda doubt it.

 
Ann

1 Like

Ann,
Thanks for the support.
What strikes me is that using the same setup, running in conjunction with SonarQube 9.6.1, it runs a lot faster, like in 6 minutes.
I usually see that scanning takes less or equal to the time it takes to build a project.
This project builds in around 11 minutes.
I have tried all sorts of configurations, including adding memory, more cores, more JVM memory, and the database on the same machine, and nothing has worked.
I also tried against the Community version (10.4) and saw faster times, but it was far from what is possible with 9.6.1.

Hi,

Again, we’ve improved analysis a fair amount since 9.6, which was released ~1.5y ago.

Analysis in Community Edition is faster because it doesn’t include taint analysis rules (altho from your log you probably don’t have those enabled) and advanced bug detection rules

Sorry, but I don’t know what else to tell you.

BTW,

Analysis doesn’t talk directly to the DB anymore. Not for a long time.

 
Ann

Ann, would you please have a last look on these 2 logs:

dev-sonar.zip (3.3 MB)
old-sonar.zip (480.8 KB)

Thank you!

Hi,


 
Ann