JavaSensor Performance

SonarQube Cloud
1

We’ve recently enabled Sonar for a legacy project within our organization and seeing slow performance in the JavaSensor, which is taking roughly 10-15 minutes. Is there any way to improve the performance?

From looking at the logs, it seems that although server-side caching is enabled, the cached data is not being used. The project has been successfully analysed on the default branch and the below logs are from a pull request. The project uses ant to build the source code.

The pipeline is running in Azure DevOps on a Microsoft-hosted agent. Would more CPUs or memory help?

2024-05-16T14:32:21.0373229Z ##[section]Starting: SonarCloudAnalyze
2024-05-16T14:32:21.0380597Z ==============================================================================
2024-05-16T14:32:21.0380759Z Task         : Run Code Analysis
2024-05-16T14:32:21.0380851Z Description  : Run scanner and upload the results to the SonarCloud server.
2024-05-16T14:32:21.0380967Z Version      : 1.45.0
2024-05-16T14:32:21.0381034Z Author       : sonarsource
2024-05-16T14:32:21.0381122Z Help         : Version: 1.45.0. This task is not needed for Maven and Gradle projects since the scanner should be run as part of the build.

[More Information](https://docs.sonarcloud.io/advanced-setup/ci-based-analysis/sonarcloud-extension-for-azure-devops/)
2024-05-16T14:32:21.0381546Z ==============================================================================
2024-05-16T14:32:21.2898155Z [command]C:\Windows\system32\cmd.exe /D /S /C "D:\a\_tasks\SonarCloudAnalyze_ce096e50-6155-4de8-8800-4221aaeed4a1\1.45.0\sonar-scanner\bin\sonar-scanner.bat"
2024-05-16T14:32:27.7998725Z INFO: Scanner configuration file: D:\a\_tasks\SonarCloudAnalyze_ce096e50-6155-4de8-8800-4221aaeed4a1\1.45.0\sonar-scanner\bin\..\conf\sonar-scanner.properties
2024-05-16T14:32:27.9693305Z INFO: Project root configuration file: NONE
2024-05-16T14:32:28.8996811Z INFO: SonarScanner 4.8.1.3023
2024-05-16T14:32:28.8997459Z INFO: Java 17.0.11 Eclipse Adoptium (64-bit)
2024-05-16T14:32:28.8997872Z INFO: Windows Server 2019 10.0 amd64
2024-05-16T14:32:39.4567067Z INFO: User cache: C:\Users\VssAdministrator\.sonar\cache
2024-05-16T14:32:48.2295259Z INFO: Analyzing on SonarCloud
2024-05-16T14:32:48.2296016Z INFO: Default locale: "en_US", source code encoding: "windows-1252" (analysis is platform dependent)
2024-05-16T14:33:02.0047717Z INFO: Load global settings
2024-05-16T14:33:02.2654427Z INFO: Load global settings (done) | time=278ms
2024-05-16T14:33:02.2694067Z INFO: Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
2024-05-16T14:33:03.9594044Z INFO: Loading required plugins
2024-05-16T14:33:03.9594376Z INFO: Load plugins index
2024-05-16T14:33:03.9594586Z INFO: Load plugins index (done) | time=125ms
2024-05-16T14:33:03.9594788Z INFO: Load/download plugins
2024-05-16T14:33:04.1202521Z INFO: Load/download plugins (done) | time=627ms
2024-05-16T14:33:04.9259644Z INFO: Found an active CI vendor: 'Azure DevOps'
2024-05-16T14:33:04.9260310Z INFO: Load project settings for component key: '<project key>'
2024-05-16T14:33:05.0141671Z INFO: Load project settings for component key: '<project key>' (done) | time=85ms
2024-05-16T14:33:05.0623365Z INFO: Process project properties
2024-05-16T14:33:05.0735551Z INFO: Project key: <project key>
2024-05-16T14:33:05.0736098Z INFO: Base dir: D:\a\1\s
2024-05-16T14:33:05.0736373Z INFO: Working dir: D:\a\1\s\.scannerwork
2024-05-16T14:33:05.0789684Z INFO: Load project branches
2024-05-16T14:33:05.1853609Z INFO: Load project branches (done) | time=107ms
2024-05-16T14:33:05.1899174Z INFO: Check ALM binding of project '<project key>'
2024-05-16T14:33:05.2540648Z INFO: Detected project binding: BOUND
2024-05-16T14:33:05.2542071Z INFO: Check ALM binding of project '<project key>' (done) | time=65ms
2024-05-16T14:33:05.2552060Z INFO: Load project pull requests
2024-05-16T14:33:05.3508961Z INFO: Load project pull requests (done) | time=96ms
2024-05-16T14:33:05.3535229Z INFO: Load branch configuration
2024-05-16T14:33:05.9862750Z INFO: Load branch configuration (done) | time=633ms
2024-05-16T14:33:06.0017527Z INFO: Load quality profiles
2024-05-16T14:33:06.2453251Z INFO: Load quality profiles (done) | time=245ms
2024-05-16T14:33:06.2537232Z INFO: Load active rules
2024-05-16T14:33:10.4478695Z INFO: Load active rules (done) | time=4175ms
2024-05-16T14:33:10.6805541Z INFO: Organization key: <organization key>
2024-05-16T14:33:10.6807726Z INFO: Pull request 648 for merge into <default branch> from <feature branch>
2024-05-16T14:33:10.7173043Z INFO: Preprocessing files...
2024-05-16T14:33:20.7194106Z INFO: 
2024-05-16T14:33:25.5662468Z WARN: File 'D:\a\1\s\<path>.xml' is bigger than 20MB and as consequence is removed from the analysis scope.
2024-05-16T14:33:26.1713925Z INFO: 3 languages detected in 6913 preprocessed files
2024-05-16T14:33:26.1718929Z INFO: 0 files ignored because of inclusion/exclusion patterns
2024-05-16T14:33:26.1722502Z INFO: 10601 files ignored because of scm ignore settings
2024-05-16T14:33:34.7879645Z INFO: Loading plugins for detected languages
2024-05-16T14:33:34.7883101Z INFO: Load/download plugins
2024-05-16T14:33:35.2400698Z INFO: Load/download plugins (done) | time=438ms
2024-05-16T14:33:35.5170563Z INFO: Load project repositories
2024-05-16T14:33:36.8607258Z INFO: Load project repositories (done) | time=1324ms
2024-05-16T14:33:36.8607901Z INFO: SCM collecting changed files in the branch
2024-05-16T14:33:37.9991103Z INFO: SCM collecting changed files in the branch (done) | time=872ms
2024-05-16T14:33:38.0399318Z INFO: Indexing files...
2024-05-16T14:33:38.0401699Z INFO: Project configuration:
2024-05-16T14:33:38.3024758Z INFO:   Excluded sources: **/build-wrapper-dump.json
2024-05-16T14:33:38.8602653Z INFO: 6913 files indexed
2024-05-16T14:33:38.8864532Z INFO: Quality profile for java: Sonar way
2024-05-16T14:33:38.8865953Z INFO: Quality profile for plsql: Sonar way
2024-05-16T14:33:38.8867996Z INFO: Quality profile for xml: Sonar way
2024-05-16T14:33:38.8868574Z INFO: ------------- Run sensors on module <project key>
2024-05-16T14:33:38.9639770Z INFO: Load metrics repository
2024-05-16T14:33:39.1729168Z INFO: Load metrics repository (done) | time=71ms
2024-05-16T14:33:39.2582415Z INFO: Sensor cache enabled
2024-05-16T14:33:40.0941332Z INFO: Load sensor cache
2024-05-16T14:33:40.8228159Z INFO: Load sensor cache (4 MB) | time=721ms
2024-05-16T14:33:43.4103256Z INFO: Sensor JavaSensor [java]
2024-05-16T14:33:43.4104489Z INFO: The Java analyzer is running in a context where unchanged files can be skipped. Full analysis is performed for changed files, optimized analysis for unchanged files.
2024-05-16T14:33:47.3305060Z INFO: Server-side caching is enabled. The Java analyzer was able to leverage cached data from previous analyses for 0 out of 4459 files. These files will not be parsed.
2024-05-16T14:33:47.3354926Z INFO: Using ECJ batch to parse 4459 Main java source files with batch size 93 KB.
2024-05-16T14:33:47.6568574Z INFO: Starting batch processing.
2024-05-16T14:36:54.7416648Z INFO: 69% analyzed
2024-05-16T14:38:06.2049734Z INFO: 100% analyzed
2024-05-16T14:38:06.2054902Z INFO: Batch processing: Done.
2024-05-16T14:38:06.2055490Z INFO: Optimized analysis for 4458 of 4459 files.
2024-05-16T14:38:09.2653160Z WARN: Dependencies/libraries were not provided for analysis of SOURCE files. The 'sonar.java.libraries' property is empty. Verify your configuration, as you might end up with less precise results.
2024-05-16T14:38:09.2992318Z WARN: Unresolved imports/types have been detected during analysis. Enable DEBUG mode to see them.
2024-05-16T14:38:09.2993195Z WARN: Use of preview features have been detected during analysis. Enable DEBUG mode to see them.
2024-05-16T14:38:09.2993585Z INFO: No "Test" source files to scan.
2024-05-16T14:38:09.2993888Z INFO: No "Generated" source files to scan.
2024-05-16T14:38:09.2994248Z INFO: Sensor JavaSensor [java] (done) | time=265921ms
2

Hi,

Welcome to the community!

That’s… not what I’m seeing in the log(?)

This looks like a 5-minute window.

But there are a couple other things in your log I want to address:

You’re using the vanilla SonarScanner CLI to anlayze Java, and an old version at that.

You should use the scanner that corresponds to how you build your project, either the SonarScanner for Gradle or the SonarScanner for Maven.

Doing so will help provide full information to analysis (with minimal additional effort on your part), and eliminate warnings like this:

 
Ann

3

Hello Ann,

Sorry, in my initial response I trimmed the log file to include only the slowest stage that I saw. Please see the attached log file that contains the entire log: sonar.txt (2.3 MB)

The project in question currently uses ant as the build tool. The SonarScanner for Ant is deprecated and therefore we are using the SonarCloud extension for Azure DevOps. I can see that this extension from Sonar is using version 4.8.1.3023. Is there another way we can use the latest version without moving to maven/gradle?

Kind Regards,

James Bodkin

4

Hi James,

Thanks for the full log. It seems that we’re looking at a large project here: 4,459 files with 108,461 functions/methods and lots of duplications. Beyond bootstrap costs, analysis duration is largely proportional to project size. This might speed up a little if you can throw more resources at it, but I doubt it would help much.

If you want to turn up the logging level, it’ s possible we might find a hotspot to address, but I kinda doubt it.

Regarding the scanner, it has always been my advice to Ant users to just exec the vanilla SonarScanner CLI, rather than trying to use the Ant task, so I suppose you’re already at the best option. On the question of version, I’ve raised the point internally to ask why we’re bundling an old scanner in the latest ADO task. Hopefully, we’ll sort that out soon.

 
Ann