SonarLint analysis slow and resource intensive for a single Typescript file

Tim_Feuerbach · June 2, 2023, 3:49pm

Operating system: Windows 10 Version 21H2
SonarLint plugin version: 8.3.0.71062
IntelliJ version: 2023.1.2
Node version: 16.20.0
NPM version: 8.19.4
Typescript version in project: 4.8.4
eslint version in project: 8.15.0
Programming language you’re coding in: Java / Typescript, but issue relates only to Typescript
Is connected mode used:
- Connected to SonarQube on premise, version 9.9 (build 65466)

Whenever I make changes to a Typescript file and the SonarLint analysis starts, my CPU fan spins up and the analysis for the single changed file is slow. I had one log that mentioned 60 seconds of analysis time.

This is not a regression, i.e. it was always slow for us. But the larger the project grows, the more noticeable it is.

Our project is organized with Nx, meaning that files are organized into “libraries”, each with their own tsconfig file which extends a base tsconfig with aliases to all the entrypoints of all the libraries. The entrypoints use index.ts “barrel” files, meaning that even when a file does not import a specific dependency, thanks to the transitive imports of the barrel file, it will be imported as well. I’m mentioning this because maybe this is the root of the problem.

Verbose output (filenames redacted with “” or removed):

Server excluded sources: 
  <redacted>
Server excluded tests: 
  <redacted>
Pinging the server
Trigger: EDITOR_CHANGE
[EDITOR_CHANGE] 1 file(s) submitted
Server excluded sources: 
  <redacted>
Server excluded tests: 
  <redacted>
Configuring analysis with org.sonarlint.intellij.java.JavaAnalysisConfigurator
Using connection '<redacted>' for project '<redacted>'
Analysing 'the-file-I-changed.ts'...
Starting analysis with configuration:
[
  projectKey: <redacted>
  baseDir: C:\<redacted>
  extraProperties: {sonar.java.target=1.5, sonar.java.libraries=<redacted>}
  moduleKey: Module: '<redacted>'
  inputFiles: [
    file:///<redacted> (UTF-8) [test]
  ]
]

  * kubernetes: 6 active rules
  * css: 23 active rules
  * scala: 30 active rules
  * kotlin: 94 active rules
  * js: 0 active rules
  * py: 206 active rules
Rule pythonsecurity:S2091 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5131 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S5633 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5135 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S2259 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S2631 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S2078 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S2076 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S2083 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5334 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S2583 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S2589 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S6465 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S6464 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S6466 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S6287 is enabled on the server, but not available in SonarLint
Rule pythonbugs:S6417 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S6350 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S3649 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5144 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5146 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5145 is enabled on the server, but not available in SonarLint
Rule pythonsecurity:S5147 is enabled on the server, but not available in SonarLint
  * secrets: 7 active rules
  * docker: 7 active rules
  * ruby: 30 active rules
  * java: 506 active rules
Rule javasecurity:S6173 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6287 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6398 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6399 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6390 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6384 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5144 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6350 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5147 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5146 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5145 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5131 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5135 is enabled on the server, but not available in SonarLint
Rule javasecurity:S2091 is enabled on the server, but not available in SonarLint
Rule javasecurity:S2083 is enabled on the server, but not available in SonarLint
Rule javasecurity:S2078 is enabled on the server, but not available in SonarLint
Rule javasecurity:S2076 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5334 is enabled on the server, but not available in SonarLint
Rule javasecurity:S5883 is enabled on the server, but not available in SonarLint
Rule javabugs:S6417 is enabled on the server, but not available in SonarLint
Rule javabugs:S6416 is enabled on the server, but not available in SonarLint
Rule javasecurity:S3649 is enabled on the server, but not available in SonarLint
Rule javabugs:S2190 is enabled on the server, but not available in SonarLint
Rule javabugs:S6322 is enabled on the server, but not available in SonarLint
Rule javabugs:S6320 is enabled on the server, but not available in SonarLint
Rule javasecurity:S2631 is enabled on the server, but not available in SonarLint
Rule javasecurity:S6096 is enabled on the server, but not available in SonarLint
  * web: 29 active rules
  * xml: 24 active rules
  * php: 197 active rules
Rule phpsecurity:S2083 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S2091 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S2631 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S2078 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5335 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S2076 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5334 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S6350 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5144 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5146 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5145 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S3649 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5131 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5135 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S6173 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S5883 is enabled on the server, but not available in SonarLint
Rule phpsecurity:S6287 is enabled on the server, but not available in SonarLint
  * terraform: 49 active rules
  * cloudformation: 26 active rules
  * swift: 62 active rules
  * ts: 134 active rules
Rule common-ts:DuplicatedBlocks is enabled on the server, but not available in SonarLint
  * yaml: 0 active rules
Start analysis
Index files
Language of file 'file:///the-file-I-changed.ts' is detected to be 'TypeScript'
1 file indexed
'JavaSensor' skipped because there is no related files in the current project
'Python Sensor' skipped because there is no related files in the current project
'Kotlin Sensor' skipped because there is no related files in the current project
'IaC Terraform Sensor' skipped because there is no related files in the current project
'IaC CloudFormation Sensor' skipped because there is no related files in the current project
'IaC Kubernetes Sensor' skipped because there is no related files in the current project
'JavaScript analysis' skipped because there is no related files in the current project
Execute Sensor: TypeScript analysis
eslint-bridge server is up, no need to start.
Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)
Initializing linter "default" with max-statements-per-line,no-nested-incdec,curly,sockets,xpath,no-commented-code,sonar-no-fallthrough,non-existent-operator,no-inverted-boolean-check,no-small-switch,no-for-in-iterable,prefer-for-of,sql-queries,no-array-delete,no-alphabetical-sort,consistent-type-assertions,arguments-order,adjacent-overload-signatures,no-unsafe-finally,prefer-while,no-sequences,standard-input,sonar-max-lines-per-function,comma-or-logical-or-case,label-position,conditional-indentation,no-use-of-empty-return-value,encryption,process-argv,super-invocation,no-try-promise,regular-expr,no-useless-increment,no-throw-literal,no-same-line-conditional,pseudo-random,no-empty-interface,no-identical-functions,no-element-overwrite,no-equals-in-for-termination,no-sparse-arrays,no-redundant-jump,prefer-readonly,no-redeclare,no-duplicate-imports,no-unthrown-error,no-globals-shadowing,bool-param-default,no-unnecessary-type-arguments,no-collection-size-mischeck,prefer-namespace-keyword,cookies,no-empty-collection,hashing,eqeqeq,no-accessor-field-mismatch,cors,no-unused-collection,no-invariant-returns,no-case-label-in-switch,no-return-await,prefer-template,no-unnecessary-type-assertion,os-command,no-implicit-dependencies,no-dead-store,no-this-alias,prefer-const,prefer-type-guard,no-return-type-any,use-type-alias,no-misleading-array-reverse,no-in-misuse,no-inconsistent-returns,no-parameter-reassignment,no-all-duplicated-branches,no-identical-conditions,updated-loop-counter,no-useless-intersection,sonar-block-scoped-var,no-extra-semi,max-switch-cases,no-non-null-assertion,deprecation,misplaced-loop-counter,no-one-iteration-loop,no-multi-str,no-undefined-argument,function-inside-loop,cognitive-complexity,max-union-size,use-isnan,no-nested-template-literals,no-duplicate-in-composite,no-ignored-return,no-caller,no-inferrable-types,no-duplicated-branches,no-redundant-parentheses,call-argument-line,no-unenclosed-multiline-block,no-duplicate-string,bitwise-operators,class-name,sonar-max-lines,max-len,code-eval,no-identical-expressions,no-redundant-boolean,prefer-immediate-return,no-useless-catch,prefer-promise-shorthand,no-template-curly-in-string,no-variable-usage-before-declaration,unused-import,no-nested-assignment,index-of-compare-to-positive-number,eol-last,variable-name,file-name-differ-from-class,no-primitive-wrappers,todo-tag,prefer-default-last,no-gratuitous-expressions,no-empty-pattern,no-self-assign,no-misused-new,no-invalid-await,no-hardcoded-credentials,fixme-tag,no-empty,sonar-max-params,no-unused-expressions
Pinging the server
Pinging the server
Found 46 tsconfig.json file(s): [<all tsconfig files of all libraries>]
tsconfig tsconfig.json files []
Adding referenced project's tsconfigs [tsconfig.app.json, tsconfig.spec.json]
<more adding tsconfig omitted>
tsconfig tsconfig.spec.json files [<all files covered by tsconfig>]
<every file in the project is being added >
Analyzing 1 files using tsconfig: e2e/tsconfig.e2e.json
1 source file to be analyzed
Cache strategy set to 'NO_CACHE' for file '[uri=the-file-I-changed.ts]' as the runtime API is not compatible
Analyzing file: fthe-file-I-changed.ts
Analyzing file "the-file-I-changed.ts" with linterId "default"
1/1 source file has been analyzed
Hit the cache for 0 out of 1
Miss the cache for 1 out of 1: RUNTIME_API_INCOMPATIBLE [1/1]
'JavaScript inside YAML analysis' skipped because there is no related files in the current project
'JavaScript inside HTML analysis' skipped because there is no related files in the current project
Execute Sensor: CSS Rules
No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
'Ruby Sensor' skipped because there is no related files in the current project
'Scala Sensor' skipped because there is no related files in the current project
'HTML' skipped because there is no related files in the current project
'XML Sensor' skipped because there is no related files in the current project
'PHP sensor' skipped because there is no related files in the current project
'Analyzer for "php.ini" files' skipped because there is no related files in the current project
Execute Sensor: TextAndSecretsSensor
1 source file to be analyzed
1/1 source file has been analyzed
'Swift Code Quality and Security' skipped because there is no related files in the current project
Execute Sensor: IaC Docker Sensor
0 source files to be analyzed
0/0 source files have been analyzed
Done in 18698ms

Processed 0 issues
Found 0 issues and 0 hotspots
Pinging the server

Looking at the speed at which the log output is printed, the majority of the time is spent on adding all the Typescript files of the project to the analysis. Only then Sonarlint sees that the file is covered by a specific tsconfig file.

I’m also curious about this line: Cache strategy set to 'NO_CACHE' for file '[uri=the-file-I-changed.ts]' as the runtime API is not compatible

Whose API is that which is incompatible?

victor.diez · June 12, 2023, 4:04pm

Hello @Tim_Feuerbach,

as for the latest question, no need to worry about the cache strategy, that’s mainly for SonarQube and SonarCloud, and the runtime API refers to SonarLint, which does not support Cache.

About the tsconfig.json files problem, it’s a non-trivial problem when dealing with projects will large amounts of tsconfig files, as the analyzer cannot know beforehand to which tsconfig.json a file belongs to.

I would advise creating a single tsconfig.json in the root of your project including all files, and only use that one in the tsconfigPath sonar property. This will override the tsconfig lookup mechanism and will force the analyzer to use the provided tsconfig file[s]. Depending on the size of the whole project it may cause memory issues (you may try increasing the node memory with the property sonar.javascript.node.maxspace increasing it to 8192 or higher), but performance should improve as it will only create one TS Program.

If a single tsconfig is not possible, I would advise creating the minimum tsconfigs needed, making each the common denominator configuration for the included files on each tsconfig file. And making sure they exclude directories that do not use their compilerOptions, to avoid having files included in more than one program. Many compilerOptions do not affect the analysis results, so it should be easier to merge tsconfigs when removing those. Any option which affects the JS output can be removed (outDir, target…). However, options affecting module detection are indeed important to properly create the program and type-check against the right dependencies (module, moduleResolution, moduleDetection, paths, lib, baseUrl…). So, if all subfolders share the same values in all of these, you may only need to use one tsconfig.

In the end, program creation for big projects takes a lot of time and the analyzer cannot avoid spending the time creating them, so any help/hint provided will help with analysis times. Creating fewer programs containing more files and properly setting files, inclusions, and exclusions will greatly help the performance. Depending on the project size, memory will then be the main issue instead.

Cheers,
Victor

Tim_Feuerbach · June 15, 2023, 7:09am

Thank you very much, creating a dedicated tsconfig file just for Sonar gravely reduced the time spent on discovery of the individual tsconfig files and their sources. We have some remaining time to shave off, but as far as we can see, that’s related to the included eslint analysis, so we will focus on removing slow eslint rules.

system · June 22, 2023, 7:09am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.