SonarScanner is VERY slow

alexo · April 8, 2024, 2:22pm

Hi,

Our SonarScanner is running very slowly. It used to work just fine, and then one day - it stopped working properly.

Here’s some information beforehand:

SonarQube 10.4.1, SonarScanner 5.0.1.3006
SonarQube deployed via SonarQube-AzureAppService with some minor modifications to make it run 10.4.1.
I’m trying to get SonarScanner to work the way it used to work.
I’ve tried several things, including searching in the forums, changing how we define new code, reducing the number of files to scan, upgrading to the new SonarScanner (previous version was 4.6.2.2472), upgrading to the new version of SonarQube (previous was the LTS 9.9). Removing the repo entirely and starting over. Nothing fixed the issue.

Scanner is running inside of a Github Action on a self-hosted runner.

Logs:

Run sonar-scanner --define sonar.cfamily.build-wrapper-output=bw-output  
INFO: Scanner configuration file: /home/user/.sonar/sonar-scanner-5.0.1.3006-linux/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/user/actions-runner/_work/repo-home/repo-home/sonar-project.properties
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.7 Eclipse Adoptium (64-bit)
INFO: Linux 5.15.0-1052-azure amd64
INFO: User cache: /home/user/.sonar/cache
INFO: Analyzing on SonarQube server 10.4.1.88267
INFO: Default locale: "en", source code encoding: "UTF-8" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=1346ms
INFO: Server id: XXXXXXXXX
INFO: User cache: /home/user/.sonar/cache
WARN: sonar.plugins.downloadOnlyRequired is false, so ALL available plugins will be downloaded
INFO: Loading all plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=511ms
INFO: Load/download plugins
INFO: Load/download plugins (done) | time=38465ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Process project properties (done) | time=90ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=19ms
INFO: Project key: XXXXXXXXX
INFO: Base dir: /home/user/actions-runner/_work/repo-home/repo-home
INFO: Working dir: /home/user/actions-runner/_work/repo-home/repo-home/.scannerwork
INFO: Load project settings for component key: 'XXXXXXXXX'
INFO: Load project settings for component key: 'XXXXXXXXX' (done) | time=271ms
INFO: Load project branches
INFO: Load project branches (done) | time=339ms
INFO: Load branch configuration
INFO: Detected branch/PR in 'GitHub Action'
INFO: Auto-configuring pull request '590'
INFO: Load branch configuration (done) | time=40ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=553ms
INFO: Auto-configuring with CI 'Github Actions'
INFO: Load active rules
INFO: Load active rules (done) | time=22717ms
INFO: Load analysis cache
INFO: Load analysis cache | time=614ms
INFO: Pull request 590 for merge into main from alexo/feature-5656
INFO: Preprocessing files...
INFO: 3 languages detected in 642 preprocessed files
INFO: 678 files ignored because of inclusion/exclusion patterns
INFO: 2 files ignored because of scm ignore settings
INFO: Load project repositories
INFO: Load project repositories (done) | time=453ms
INFO: SCM collecting changed files in the branch
INFO: Merge base sha1: 2a1b3ddabb1b3c762dbce7ba14de5e1e657a4ab9
INFO: SCM collecting changed files in the branch (done) | time=9861ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Excluded sources: FreeRTOS/**, .idea/**, cmake-*/**
INFO:   Excluded sources for coverage: **/*.*
INFO: 642 files indexed
INFO: Quality profile for c: Sonar way
INFO: Quality profile for json: Sonar way
INFO: Quality profile for py: Sonar way
INFO: ------------- Run sensors on module XXXXXXXXX
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=240ms
INFO: Sensor IaC CloudFormation Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC CloudFormation Sensor [iac] (done) | time=532ms
INFO: Sensor IaC AzureResourceManager Sensor [iac]
INFO: Sensor IaC AzureResourceManager Sensor is restricted to changed files only
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC AzureResourceManager Sensor [iac] (done) | time=875ms
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=50ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Sensor C# Analysis Log [csharp] (done) | time=172ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=1ms
INFO: Sensor HTML [web]
INFO: Sensor HTML is restricted to changed files only
INFO: Sensor HTML [web] (done) | time=40ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: Sensor TextAndSecretsSensor is restricted to changed files only
INFO: 14 source files to be analyzed
INFO: 7/14 files analyzed, current file: XXXXXXXXXXXXXXXXXX.c
INFO: 14/14 source files have been analyzed
INFO: Sensor TextAndSecretsSensor [text] (done) | time=24820ms
INFO: Sensor VB.NET Project Type Information [vbnet]
INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=5ms
INFO: Sensor VB.NET Analysis Log [vbnet]
INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=142ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
INFO: Sensor Python Sensor [python]
WARN: Your code is analyzed as compatible with all Python 3 versions by default. You can get a more precise analysis by setting the exact Python version in your configuration via the parameter "sonar.python.version"
INFO: Using cached data to retrieve global symbols.
INFO: Cached information of global symbols will be used for 15 out of 16 main files. Global symbols will be recomputed for the remaining files.
INFO: Fully optimized analysis can be performed for 15 out of 16 files.
INFO: Partially optimized analysis can be performed for 15 out of 16 files.
INFO: Starting global symbols computation
INFO: 1 source file to be analyzed
INFO: 1/1 source file has been analyzed
INFO: Starting rules execution
INFO: 16 source files to be analyzed
INFO: 0/16 files analyzed, current file: XXXXXXXXXXXXXXXXXX.py
INFO: 12/16 files analyzed, current file: XXXXXXXXXXX.py
INFO: 12/16 files analyzed, current file: XXXXXXXXXXXXX.py
INFO: 16/16 source files have been analyzed
INFO: The Python analyzer was able to leverage cached data from previous analyses for 15 out of 16 files. These files were not parsed.
INFO: Sensor Python Sensor [python] (done) | time=46116ms
INFO: Sensor Cobertura Sensor for Python coverage [python]
INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=2260ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=1869ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=29ms
INFO: Sensor CSS Rules [javascript]
INFO: Sensor CSS Rules is restricted to changed files only
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=10ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=21ms
INFO: Sensor Python HTML templates processing [securitypythonfrontend]
INFO: HTML files are not indexed : you may want to add them in the scanned files of this project to detect Python XSS vulnerabilities
INFO: Sensor Python HTML templates processing [securitypythonfrontend] (done) | time=82ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: Sensor IaC Docker Sensor is restricted to changed files only
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=353ms
INFO: Sensor Serverless configuration file sensor [security]
INFO: 0 Serverless function entries were found in the project
INFO: 0 Serverless function handlers were kept as entrypoints
INFO: Sensor Serverless configuration file sensor [security] (done) | time=20ms
INFO: Sensor AWS SAM template file sensor [security]
INFO: Sensor AWS SAM template file sensor [security] (done) | time=20ms
INFO: Sensor AWS SAM Inline template file sensor [security]
INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=10ms
INFO: Sensor CFamily [cpp]
INFO: CFamily plugin version: 6.52.1.66618
INFO: Using build-wrapper output: /home/user/actions-runner/_work/repo-home/repo-home/bw-output/build-wrapper-dump.json
INFO: Available processors: 2
INFO: Using 2 threads for analysis.
WARN: Invalid character encountered in file /home/user/actions-runner/_work/repo-home/repo-home/XXXXXXXXXXX.h at line 41 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
WARN: Invalid character encountered in file /home/user/actions-runner/_work/repo-home/repo-home/XXXXXXXXXXXXXXXXXX.h at line 74 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
WARN: Invalid character encountered in file /home/user/actions-runner/_work/repo-home/repo-home/XXXXXXXXXXXX.h at line 2 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
INFO: Loading cache from: server
>>>>>>> Here each line takes like a minute
INFO: [pool-6-thread-1] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX1.c
INFO: [pool-6-thread-1] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX2.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX3.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX4.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX5.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX6.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX7.c
INFO: [pool-6-thread-2] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX8.c
INFO: [pool-6-thread-1] /home/sensos/actions-runner/_work/repo-home/repo-home/app_middleware/fileXXXXX9.c

mpaladin · April 10, 2024, 8:14am

Hi @alexo ,

could you try to rerun the analysis with sonar.cfamily.threads=1 property?

alexo · April 17, 2024, 3:04pm

I tried that and did not see an improvement

mpaladin · April 17, 2024, 3:39pm

Hi @alexo ,

could you please share the full sonar-scanner log in verbose mode? It would be great if you could get the log with timestamp from GitHub actions. Do you want me to send you a private message where you can share it privately?

alexo · April 18, 2024, 8:16am

Yes, that’d be great. Also, please help as I don’t really know how to set up sonar-scanner in verbose mode.

mpaladin · April 18, 2024, 4:07pm

Hi @alexo ,

you can pass --debug to the sonar-scanner. Sending you a private message.