React component analysis takes 45 minutes instead of 2

[Mark_Gortzak]

Hi,

Since Friday afternoon we (too) are noticing that the SonarCloud anlysis for our React project is suddenly very slow. For about ~3000 files we went from a duration of ~2 minutes to ~45 minutes. This severly impacts our development lifecycle.

We are running Bitbucket pipelines, and using the maven sonar-scanner to analyse the React code.

Before the logs looked like this:

[INFO] Sensor TypeScript analysis [javascript]
[INFO] Resolving TSConfig files using './tsconfig.json' from property sonar.typescript.tsconfigPath
[INFO] Found 1 TSConfig file(s): [/opt/atlassian/pipelines/agent/build/foo/bar/./tsconfig.json]
[INFO] Creating TypeScript program
[INFO] TypeScript configuration file /opt/atlassian/pipelines/agent/build/foo/bar/./tsconfig.json
[INFO] 2939 source files to be analyzed
...
[INFO] 2939/2939 source files have been analyzed
[INFO] Hit the cache for 0 out of 2905
[INFO] Miss the cache for 2905 out of 2905: ANALYSIS_MODE_INELIGIBLE [2905/2905]
[INFO] Sensor TypeScript analysis [javascript] (done) | time=95926ms

Analysis took less than 2 minutes. This build was triggered on Friday the 16th on 16:00 GMT+2.

On Monday we saw that analysis got very slow:

[INFO] Sensor JavaScript/TypeScript analysis [javascript]
[INFO] 2956 source files to be analyzed
[INFO] Resolving TSConfig files using './tsconfig.json' from property sonar.typescript.tsconfigPath
[INFO] Found 1 TSConfig file(s): [/opt/atlassian/pipelines/agent/build/foo/bar/./tsconfig.json]
...
[INFO] 2956/2956 source files have been analyzed
[INFO] Hit the cache for 0 out of 2956
[INFO] Miss the cache for 2956 out of 2956: ANALYSIS_MODE_INELIGIBLE [2956/2956]
[INFO] Sensor JavaScript/TypeScript analysis [javascript] (done) | time=2328248ms

Analysis took ~38 minutes. Which is actually a very “fast” slow build. It is still 19 (!) times as slow as it previous was.

Our root tsconfig.json looks like this:

{
  "references": [
    {
      "path": "./packages/package1"
    },
    {
      "path": "./packages/package2"
    }
  ],
  "compilerOptions": {
  	...
  },
  "include": [
    "./src/**/*.js",
    "./src/**/*.ts",
    "./src/**/*.tsx"
  ],
  "exclude": [
    "node_modules/"
  ]
}

SonarCloud itself is configured like this in the pom.xml (which resides besides the root tsconfig.json):

<properties>
    <sonar.sources>pom.xml,package.json,src/,packages/</sonar.sources>
    <sonar.typescript.tsconfigPath>./tsconfig.json</sonar.typescript.tsconfigPath>
</properties>

So the first thing I did was looking for solutions on this very forum. I found this issue with a suggested work-around:

typescript:S4328 suddenly detecting internal modules

Hi @aporter-varicent, Seems the solution that worked for @stevendarby (via PM) was to set sonar.typescript.tsconfigPaths with the tsconfig.base.json in the project root. Can you check if that works for you? Cheers, Victor

Which I did in the pom.xml:

<properties>
    <sonar.sources>pom.xml,package.json,src/,packages/</sonar.sources>
    <sonar.typescript.tsconfigPaths>./tsconfig.json</sonar.typescript.tsconfigPaths>
</properties>

The logs looked like this:

[INFO] 17:14:24.540 Sensor JavaScript/TypeScript analysis [javascript]
[DEBUG] 17:14:24.543 eslint-bridge server is up, no need to start.
[INFO] 17:14:24.543 2964 source files to be analyzed
[DEBUG] 17:14:24.544 Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)
[INFO] 17:14:24.544 Resolving TSConfig files using './tsconfig.json' from property sonar.typescript.tsconfigPaths
[DEBUG] 17:14:24.544 Using './tsconfig.json' to resolve TSConfig file(s)
[INFO] 17:14:24.545 Found 1 TSConfig file(s): [/opt/atlassian/pipelines/agent/build/foo/bar/./tsconfig.json]
...
[INFO] 17:59:56.248 2964/2964 source files have been analyzed
[INFO] 17:59:56.248 Hit the cache for 0 out of 2964
[INFO] 17:59:56.248 Miss the cache for 2964 out of 2964: ANALYSIS_MODE_INELIGIBLE [2964/2964]
[INFO] 17:59:56.248 Sensor JavaScript/TypeScript analysis [javascript] (done) | time=2731708ms

Analysis took ~45 minutes. So that work-around didn’t work for us .

The next thing I tried was this:

SonarCloud scan is taking very long time

Hello @ianhomer, thanks for your feedback. Indeed last changes seems to have affected projects with several tsconfigs. Can you please confirm that’s your case? One workaround until we provide a fix is to provide the sonar property sonar.typescript.tsconfigPath pointing to the base tsconfig in the project root. Let me know if that helps in your case. About the logs, if you enable debug logs (-X) you will see that the analyzer is still finding the tsconfig files, it’s just following a different …

This is not a monorepo, but you never know.

The logs looked like this:

INFO] 09:38:58.996 Sensor JavaScript/TypeScript analysis [javascript]
[DEBUG] 09:38:59.000 eslint-bridge server is up, no need to start.
[INFO] 09:38:59.000 2964 source files to be analyzed
[DEBUG] 09:38:59.001 Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)
[INFO] 09:38:59.001 Resolving TSConfig files using 'tsconfig.json' from property sonar.typescript.tsconfigPath
[DEBUG] 09:38:59.001 Using 'tsconfig.json' to resolve TSConfig file(s)
[INFO] 09:38:59.002 Found 1 TSConfig file(s): [/opt/atlassian/pipelines/agent/build/foo/bar/tsconfig.json]
...
[INFO] 10:22:44.637 Hit the cache for 0 out of 2964
[INFO] 10:22:44.637 Miss the cache for 2964 out of 2964: ANALYSIS_MODE_INELIGIBLE [2964/2964]
[INFO] 10:22:44.637 Sensor JavaScript/TypeScript analysis [javascript] (done) | time=2625641ms

Analysis took ~43 minutes. This work-around didn’t work either .

Because this bug more than doubled our build time we disabled analysis in the pom.xml for now:

<properties>
    <sonar.sources>pom.xml,package.json,src/,packages/</sonar.sources>
    <sonar.typescript.tsconfigPath>./tsconfig.json</sonar.typescript.tsconfigPath>

    <!-- TODO skip SonarCloud analysis for now because it takes ~45 minutes instead of ~2 minutes to scan this module -->
    <sonar.skip>true</sonar.skip>
    <!-- TODO remove this property when SonarCloud works properly again -->
</properties>

For now this is the only working work-around for us.

Now, my question is (ofc) what can I do to actually make this work again?

I saw an issue where a dedicated tsconfig.json specifically for SonarCloud worked:

SonarLint analysis slow and resource intensive for a single Typescript file

Hello @Tim_Feuerbach, as for the latest question, no need to worry about the cache strategy, that’s mainly for SonarQube and SonarCloud, and the runtime API refers to SonarLint, which does not support Cache. About the tsconfig.json files problem, it’s a non-trivial problem when dealing with projects will large amounts of tsconfig files, as the analyzer cannot know beforehand to which tsconfig.json a file belongs to. I would advise creating a single tsconfig.json in the root of your project in…

I’d like to try that, we do need some help with that. I couldn’t find any documentation on what an optimized tsconfig.json looks like. Might that be a solution?

And an other question I have is when can we expect a proper fix for this problem?

If needed I can seen debug logging for the not working work-arounds, but only in a PM.

[victor.diez]

Hi @Mark_Gortzak,

I’ll ask you for the logs in private then.

Thanks,
Victor

[Mark_Gortzak]

I just send you the logs.

[Mark_Gortzak]

After some help from @victor.diez it is working again:

[INFO] 15:21:27.180 Sensor JavaScript/TypeScript analysis [javascript]
[DEBUG] 15:21:27.182 eslint-bridge server is up, no need to start.
[INFO] 15:21:27.183 2964 source files to be analyzed
[DEBUG] 15:21:27.184 Analysis of unchanged files will not be skipped (current analysis requires all files to be analyzed)
[INFO] 15:21:27.184 Resolving TSConfig files using 'tsconfig.sonar.json' from property sonar.typescript.tsconfigPath
[DEBUG] 15:21:27.184 Using 'tsconfig.sonar.json' to resolve TSConfig file(s)
[INFO] 15:21:27.184 Found 1 TSConfig file(s): [/opt/atlassian/pipelines/agent/build/foo/bart/tsconfig.sonar.json]
...
[INFO] 15:23:28.740 2964/2964 source files have been analyzed
[INFO] 15:23:28.740 Hit the cache for 0 out of 2964
[INFO] 15:23:28.741 Miss the cache for 2964 out of 2964: ANALYSIS_MODE_INELIGIBLE [2964/2964]
[INFO] 15:23:28.741 Sensor JavaScript/TypeScript analysis [javascript] (done) | time=121561ms

Analysis took ~2 minutes again.

This work-around entailed creating a separate tsconfig.json and only have compilerOptions in it. In my example:

1. I copied the existing tsconfig.json to tsconfig.sonar.json
2. Remove everything in tsconfig.sonar.json except the compilerOptions
3. Change in pom.xml the value of the sonar.typescript.tsconfigPath property to tsconfig.sonar.json.

My tsconfig.sonar.json looks like this:

{
  "compilerOptions": {
    "esModuleInterop": true,
    "lib": [
      "es2015",
      "dom"
    ],
    "moduleResolution": "node",
    "jsx": "react-jsx",
    "strict": false,
    "allowJs": true,
    "checkJs": false,
    "noEmit": true,
    "target": "esnext",
    "skipLibCheck": true,
    "allowSyntheticDefaultImports": true,
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "typeRoots": [
      "./node_modules/@types",
      "./src/types/"
    ],
    "noFallthroughCasesInSwitch": true,
    "declaration": true,
    "outDir": "dist",
    "strictBindCallApply": true
  }
}

The property allowJs should be set to true if you analyze JavaScript files.