SonarCloud JsSecuritySensor is very slow recently

Wen_Han_Chang · July 28, 2023, 10:19am

ALM used: GitLab
CI system used: GitLab CI
Scanner command used when applicable: sonar-scanner
Languages of the repository: TypeScript/JavaScript (most are TS)
LOC: ~150k lines

Hi, I’m currently facing an issue with SonarCloud scanning within my repository. Recently, the scan duration has ballooned from approximately 5 minutes to a considerably slower 20-30 minutes. Upon review, it appears that the JsSecuritySensor is the main culprit for the additional time spent.

The SonarCloud logs show the following before getting stuck:

INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: /my_repo/[MASKED]/.scannerwork/ucfg2/js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /my_repo/[MASKED]/.scannerwork/ucfg2/js
INFO: Cache loaded: SecuritySensorCache{ucfgsHashes.count=14102, typeDefinitionsHashes.count=0}
INFO: 08:18:02.416565 Building Runtime Type propagation graph
INFO: 08:18:03.583675 Running Tarjan on 195140 nodes
INFO: 08:18:03.838945 Tarjan found 195136 components
INFO: 08:18:04.299394 Variable type analysis: done
INFO: 08:18:04.310375 Building Runtime Type propagation graph
INFO: 08:18:05.654968 Running Tarjan on 195140 nodes
INFO: 08:18:05.812659 Tarjan found 195136 components
INFO: 08:18:06.177978 Variable type analysis: done
INFO: Analyzing 14086 ucfgs to detect vulnerabilities.
INFO: Taint analysis starting. Entrypoints: 2541
INFO: Running symbolic analysis for 'JS'

It then pauses for approximately 15 minutes before completing:

INFO: Taint analysis: done.
INFO: Sensor JsSecuritySensor [security] (done) | time=939479ms

For comparison, here are logs from several weeks ago (June 14), where the JsSecuritySensor process was significantly faster:

INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: /my_repo/[MASKED]/.scannerwork/ucfg2/js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /my_repo/[MASKED]/.scannerwork/ucfg2/js
INFO: 09:17:11.472807 Building Runtime Type propagation graph
INFO: 09:17:11.961254 Running Tarjan on 100953 nodes
INFO: 09:17:12.078573 Tarjan found 100949 components
INFO: 09:17:12.258211 Variable type analysis: done
INFO: 09:17:12.261047 Building Runtime Type propagation graph
INFO: 09:17:12.764111 Running Tarjan on 100953 nodes
INFO: 09:17:12.838845 Tarjan found 100949 components
INFO: 09:17:12.991821 Variable type analysis: done
INFO: Analyzing 13775 ucfgs to detect vulnerabilities.
INFO: Taint analysis starting. Entrypoints: 2483
INFO: Running symbolic analysis for 'JS'
INFO: Taint analysis: done.
INFO: Sensor JsSecuritySensor [security] (done) | time=15999ms

Despite the very similar logs, the execution time has ballooned from 16 seconds to 15 minutes.

I’ve browsed the forum and attempted the following solutions:

Add TypeScript tsconfig.json location to project (Slow Automatic Analysis of Javascript/Typescript - #7 by victor.diez)
Deactive all the “Vulnerability” rules in TypeScript and JavaScript (Sonar scanner has started to take too much time to run analysis)

Unfortunately, neither approach has improved the situation; the scanning time remains at around 20-30 minutes. Additionally, I attempted to use debug mode (sonar-scanner -x), but this created such an extensive output that GitLab CI couldn’t display it in its entirety.

Any suggestions or insights on how to address this problem would be greatly appreciated.

Wen_Han_Chang · August 1, 2023, 4:27am

Update:

Here are the new scenarios I’ve tried:

Disable all TypeScript and JavaScript rules:

The JavaScript/TypeScript analysis took a reasonable 15 seconds.

INFO: Sensor JavaScript/TypeScript analysis [javascript]
INFO: Resolving TSConfig files using 'tsconfig.json' from property sonar.typescript.tsconfigPaths
INFO: Found 1 TSConfig file(s): [/my_repo/[MASKED]/tsconfig.json]
INFO: Creating TypeScript program
INFO: TypeScript configuration file /my_repo/[MASKED]/tsconfig.json
INFO: 2544 source files to be analyzed
INFO: Creating TypeScript program (done) | time=4134ms
INFO: Starting analysis with current program
INFO: 39/2544 files analyzed, current file: /my_repo/[MASKED]/foobar.ts
INFO: Analyzed 2553 file(s) with current program
INFO: Found 1 file(s) not part of any tsconfig.json: they will be analyzed without type information
INFO: 2544/2544 source files have been analyzed
INFO: Hit the cache for 2498 out of 2554
INFO: Miss the cache for 56 out of 2554: FILE_CHANGED [56/2554]
INFO: Sensor JavaScript/TypeScript analysis [javascript] (done) | time=15419ms

And JsSecuritySensor process completed immediately, indicating no analysis was conducted.

INFO: Sensor JsSecuritySensor [security]
INFO: No taint analysis rules have been enabled, will not execute taint analysis.
INFO: Sensor JsSecuritySensor [security] (done) | time=0ms

Disable all TypeScript and JavaScript rules, but activate only one security-related rule (tssecurity:S5883):

Again, the JavaScript/TypeScript analysis took a consistent 15 seconds. However, the JsSecuritySensor process took a full 15 minutes to complete, even though only one rule was active. This is the same time it took when all rules were active.

INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: /my_repo/[MASKED]/.scannerwork/ucfg2/js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: /my_repo/[MASKED]/.scannerwork/ucfg2/js
INFO: Cache loaded: SecuritySensorCache{ucfgsHashes.count=14102, typeDefinitionsHashes.count=0}
INFO: 03:42:00.926202 Building Runtime Type propagation graph
INFO: 03:42:02.023764 Running Tarjan on 195622 nodes
INFO: 03:42:02.254466 Tarjan found 195618 components
INFO: 03:42:02.589999 Variable type analysis: done
INFO: 03:42:02.593401 Building Runtime Type propagation graph
INFO: 03:42:03.973421 Running Tarjan on 195622 nodes
INFO: 03:42:04.1656 Tarjan found 195618 components
INFO: 03:42:04.621221 Variable type analysis: done
INFO: Analyzing 14149 ucfgs to detect vulnerabilities.
INFO: Taint analysis starting. Entrypoints: 2554
INFO: Running symbolic analysis for 'JS'
INFO: Taint analysis: done.
INFO: Sensor JsSecuritySensor [security] (done) | time=919181ms

The time consumption of the JsSecuritySensor process does not seem to correlate with the number of active rules. Even with just a single active rule, it still takes a significant amount of time to complete. This suggests that the issue might lie within the JsSecuritySensor itself.

For the time being, my workaround is to disable all the “Vulnerability” and “Security Hotspot” rules in my project to bypass the JsSecuritySensor. However, this is not a sustainable solution, as it can compromise the security checks for the project.

I would appreciate it if you guys could look into this issue with the JsSecuritySensor. Any insights or suggestions on how to address this problem would be greatly appreciated.

czurn · August 3, 2023, 12:39pm

Hello, and thank you for reporting this.

I’ve created an internal ticket in order to investigate this issue: such a jump in analysis time is indeed very surprising.
Is the project you are analyzing open source by any chance? So that we can reproduce the issue on our side. If not, would you be willing to share the content of the ucfg2/js folder with us privately? That would also help us by allowing us to reproduce the security analysis part on our end.

Regarding the deactivated rules, indeed, this can work as a temporary workaround in the meantime. Note that you can reactivate the security hotspot rules and some of the vulnerabilities you have disabled. The JsSecuritySensor is related to injection rules, so deactivating only vulnerability injection rules would be enough to skip the analysis of the JsSecuritySensor. You can find all the vulnerability injection rules for Javascript (and Typescript) here: JavaScript static code analysis | injection.

Best,
-Christophe

johnyborek · September 4, 2023, 3:14pm

Hello,
I have exactly the same problem, the execution time of SonarCloud analysis increased from around 4-5 minutes to around 23-24 minutes for my project.
It was working ok on 17 July, we started getting these slow runs from 18 July.
What is interesting, that the number of elements on which the analysis is performed seems to be doubled between those two runs, although there were almost no changes to our codebase on that day.

Here’s the log for the run from 17 July:

2023-07-17T09:43:30.9897329Z INFO: 11:43:30.987107089 Building Runtime Type propagation graph
2023-07-17T09:43:32.0740209Z INFO: 11:43:32.072092494 Running Tarjan on 69967 nodes
2023-07-17T09:43:32.2577388Z INFO: 11:43:32.256604708 Tarjan found 69949 components
2023-07-17T09:43:32.6474250Z INFO: 11:43:32.645865303 Variable type analysis: done
2023-07-17T09:43:32.6511699Z INFO: 11:43:32.650591542 Building Runtime Type propagation graph
2023-07-17T09:43:33.8979256Z INFO: 11:43:33.896959271 Running Tarjan on 69967 nodes
2023-07-17T09:43:33.9761808Z INFO: 11:43:33.975081912 Tarjan found 69949 components
2023-07-17T09:43:34.2071080Z INFO: 11:43:34.206188109 Variable type analysis: done
2023-07-17T09:43:34.2176067Z INFO: Analyzing 8852 ucfgs to detect vulnerabilities.
2023-07-17T09:43:40.5641067Z INFO: Taint analysis starting. Entrypoints: 1651

And here’s the log from 18 July:

2023-07-18T11:35:31.1903694Z INFO: 13:35:31.188292576 Building Runtime Type propagation graph
2023-07-18T11:35:34.2476289Z INFO: 13:35:34.245361725 Running Tarjan on 152280 nodes
2023-07-18T11:35:34.5879947Z INFO: 13:35:34.586999605 Tarjan found 152262 components
2023-07-18T11:35:35.3503676Z INFO: 13:35:35.349561216 Variable type analysis: done
2023-07-18T11:35:35.3532967Z INFO: 13:35:35.352662865 Building Runtime Type propagation graph
2023-07-18T11:35:36.7502486Z INFO: 13:35:36.749438164 Running Tarjan on 152280 nodes
2023-07-18T11:35:36.8682743Z INFO: 13:35:36.867443923 Tarjan found 152262 components
2023-07-18T11:35:37.3859547Z INFO: 13:35:37.385221878 Variable type analysis: done
2023-07-18T11:35:38.1273466Z INFO: Analyzing 8870 ucfgs to detect vulnerabilities.
2023-07-18T11:35:52.2078957Z INFO: Taint analysis starting. Entrypoints: 1668

Did anything change on SonarCloud side on that day?

Best regards,
Rafal Borek

johnyborek · September 4, 2023, 3:39pm

There’s also an increase in the JavaScript/TypeScript Sensor runtime. On 17 July it was taking around 180 - 190 seconds, and on 18 July it started taking around 240-250 seconds. So the problem is not as serious as for JsSecuritySensor, which takes almost 20 minutes now. But still, when you compare 180s to 240s, it’s a 33% increase in the JavaScript/TypeScript Sensor execution time, which is quite a lot.

Karim_El_Ouerghemmi · September 5, 2023, 1:14pm

Hey @johnyborek, thank you for sharing with us.

It would be nice if you could open a separate thread for the problem on your side. We’ll get back to you there.

Thank you!

Peter_Jones · February 13, 2024, 7:34pm

We are having the same issue. It’s taking 20 minutes to Read the UCFG’s.

Is there any progress on this issue?

We use SonarQube * Enterprise Edition * Version 9.9

I disabled all js and typescript rules - no difference.

monitz87 · March 4, 2024, 6:07pm

Same problem here. It takes orders of magnitude longer to read UCFGs than it did before. I wouldn’t be able to say exactly when this started happening though, but it’s definitely making us reconsider using sonarcloud because it’s really slowing our development down

It’s been 32 minutes and counting in this last run

renaud.tognelli · March 28, 2024, 10:56am

Hello @monitz87,

thank you for sharing this with us.

Would you mind opening a new thread to avoid mixing issues?

Many thanks in advance

skayal · April 30, 2024, 7:40pm

HI team,
Is there any resolution to this. We are sailing on the same boat.
We use SonarQube * Enterprise Edition * Version 9.9