SonarCloud scan is taking very long time

SonarCloud
,
1
  • ALM used - GitHub
  • CI system used - GH actions
  • Languages of the repository - ts

We have sonarCloud configured for our nx monorepo (sonar.typescript.tsconfigPath is pointing to tsconfig in the monorepo root)
We trigger the scan via our GitHub actions workflow using the action: SonarSource/sonarcloud-github-action@master.
Until a few days ago our setup was working as expected, we are now experiencing a huge performance degradation:

  • scans on PR (one line modified) takes hrs - the scan tries to analyse more then 5k files and the analysis for each files takes up to 5 seconds.
  • we noticed that the JavaScript/TypeScript/CSS Code Quality and Security plugin version in use has changed since our last performant scan (version in use is now: 10.3.1.21905).
  • our sonarCloud cache is enabled, but it looks that none of the 5k files that the scan is trying to analyse is retrieved from the cache.

Is the analysis logic changed in the last version of the package so that our cache has been invalidated or that our configuration is somehow not optimal anymore?

Could you guys suggest some paths for investigation?

Many thanks!

2 Likes
2

Hello @ElisaDellaC,

indeed cache is invalidated on each new JS/TS analyzer release. So it’s normal that the first scan will take longer.

Can you please enable debug logs (-X option) and paste here the logs? If you can provide them with and without the sonar.typescript.tsconfigPath that would help us better understand if the tsconfig resolution is affecting your results.

Thanks,
Victor

1 Like
4

Hey Victor,
thank you so much for your quick reply. I’ll produce the needed logs (partial - analysis does not look to complete even after hours that is running). Is there a way to send them to you privately?

Thanks a lot
Elisa

5

Let’s continue the conversation via PM

6

Hi, we’ve encountered a similar issue, started for us about 6pm on Friday UK time. What used to take 2 minutes now takes 45 minutes.

We enabled debug logs and noted that the sonar-scanner-cli is downloading plugins from https://sonarcloud.io/api/plugins/installed. In particular it’s installing “JavaScript/TypeScript/CSS Code Quality and Security” plugin (10.3.1 (build 21905)) which was updated at 1686939214766 Fri Jun 16 2023 18:13:34.

We’ve also notice that the analyze step in the scan is taking much longer than it used

INFO: Sensor JavaScript/TypeScript analysis [javascript]
INFO: 2242 source files to be analyzed

When it was running fast we noticed that the log indicate that it found tsconfig files and was “Creating TypeScript program”. Now it is running slow we don’t see these log messages,

Many thanks and if there is anything else I can try, to gather more information, I’ll be happy to help,

Ian

7

Hello @ianhomer,

thanks for your feedback. Indeed last changes seems to have affected projects with several tsconfigs. Can you please confirm that’s your case? One workaround until we provide a fix is to provide the sonar property sonar.typescript.tsconfigPath pointing to the base tsconfig in the project root. Let me know if that helps in your case.

About the logs, if you enable debug logs (-X) you will see that the analyzer is still finding the tsconfig files, it’s just following a different strategy assigning them to each source file.

Cheers,
Victor

2 Likes
8

Many thanks Victor - spot on.

We’re in a mono repo and have several tsconfig files. The scan is much quicker with

sonar.typescript.tsconfigPath = tsconfig.json

we see it scanning through the following now in a minute (where it was taking 45 minutes before).

INFO: Sensor JavaScript/TypeScript analysis [javascript]
INFO: 2242 source files to be analyzed
INFO: Resolving TSConfig files using 'tsconfig.json' from property sonar.typescript.tsconfigPath
INFO: Found 1 TSConfig file(s): [/app/tsconfig.json]
INFO: 220/2242 files analyzed, current file: /x/foo.ts
INFO: 732/2242 files analyzed, current file: /x/bar.ts
INFO: 1179/2242 files analyzed, current file: /x/fez.ts
INFO: 1730/2242 files analyzed, current file: /x/baz.tsx
INFO: 2242/2242 source files have been analyzed

Doing a few more tests on this, but thank you so much for your help,

Ian

1 Like
9

Hello @ElisaDellaC @ianhomer,

what kind of project you are working on? react? angular? are you using nx? we are trying to narrow the issue.

Thanks,
Victor

10

Hello Victor,
thanks a lot for your help: having a sonar.typescript.tsconfigPath pointing to the a tsconfig in the monorepo root that includes all files fixes the issue for us.

We work with an Nx monorepo with Angular and Qwik apps, if we can help further narrowing down the issue just let us know.

Cheers
Elisa

1 Like
11

I am having this same issue, my repo used to take 2 minutes, now 50 mins and I have even less files than ianhomer, we only have 1344 files.
I have set the base tsconfig on the root of our monorepo but no change on slow run of analysis.

12

Hi @jcalderonzumba,

thanks for the feedback. Are you using the sonar.typescript.tsconfigPath to point to the base tsconfig? Can you share the contents of that tsconfig? You can send them privately if you prefer.

Cheers,
Victor

13

We’re React, node back end, with npm workspaces. We have about 20 tsconfig files. 1 in root and then others in package sub directories. We also have a few alternative names for tsconfigs, e.g. tsconfig.test.json.

We’ve not noticed any side effects (yet) from using the root tsconfig for the scanning, all been working well for the last day or so.

1 Like
14

Sure no problem.

My setting is:

sonar.typescript.tsconfigPath=**/packages/tsconfig.json

And then each tsconfig.json is similar to:

{
  "extends": "../../tsconfig.build.json",
  "compilerOptions": {
    "strict": false,
    "useUnknownInCatchVariables": false,
    "outDir": "./dist",
    "rootDir": "./src",
    "baseUrl": "./",
    "composite": true,
    "allowJs": true,
    "resolveJsonModule": true
  },
  "references": [
    {
      "path": "../api-server"
    },
    {
      "path": "../logger"
    },
    {
      "path": "../sdks"
    },
    {
      "path": "../utils"
    }
  ],
  "include": ["src/**/*", "src/**/*.json"],
  "exclude": []
}

I activated debug and i can share this:

test_container | 08:39:36.591 DEBUG: Starting server (done) | time=2152ms
test_container | 08:39:36.592 INFO: 1344 source files to be analyzed

test_container | 08:39:36.596 DEBUG: Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)

test_container | 08:39:36.597 INFO: Resolving TSConfig files using '**/packages/tsconfig.json' from property sonar.typescript.tsconfigPath

test_container | 08:39:36.597 DEBUG: Using '**/packages/tsconfig.json' to resolve TSConfig file(s)

I think the log in bold is the reason why it goes super slow, it analyses everything!

15

Hi @jcalderonzumba,

can you please use just your root tsconfig see if that helps? I understand from the extends is should be tsconfig.build.json:

sonar.typescript.tsconfigPath=tsconfig.build.json

Let me know if that helps.

Cheers,
Victor

16

That log is ok, no need to worry about it. Analysis of unchanged files is only skipped on PRs.

17

It has improved a bit.

From 30ish mins to 18 to 22

test_container | INFO: Total time: 18:43.451s

Still almost 800% slower than before when it took just like 3 mins.

18

Since last week the analysis of our React repo now take 15 minutes instead of 3 mins,

Please can this update be fixed or reversed.

For reference our repo is a standard Javascript repo.

19

Hi @Paul_Rankin,

being a JS project, I understand your code base has no tsconfig files?

Victor

20

Hi @Paul_Rankin,

As a workaround, can you please create a tsconfig.sonar.json in the root of your project with the following contents?

{
    compilerOptions: {
      allowJs: true,
      noImplicitAny: true,
    },
  }

Then, set the property sonar.typescript.tsconfigPath=tsconfig.sonar.json in the analysis.

Please let me know if that helps.

Cheers,
Victor

1 Like
21

If it helps others.

I have a monorepo and i had the usual tsconfig.json and tsconfig.build.json i mixed both together and created
tsconfig.sonar.json with:

"compilerOptions": {
    "composite": true,
    "declaration": true,
    "moduleResolution": "node",
    "lib": ["es6", "DOM", "dom.iterable"],
    "target": "es6",
    "experimentalDecorators": true,
    "module": "commonjs",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strictPropertyInitialization": false,
    "strict": true,
    "noImplicitAny": true,
    "skipLibCheck": true,
    "baseUrl": "./",
    "paths": {
      "@payflow/*": ["packages/*/src"]
    }
  },

And added sonar.typescript.tsconfigPath=tsconfig.sonar.json that got back my repo to 3 mins as it used to be

2 Likes