Sonarqube scanner perf degradation after switching to pnpm

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    SonarScanner 5.0.1.3006
    Java 17.0.8 Alpine (64-bit)
    Linux 5.15.0-1030-gcp amd64

  • what are you trying to achieve]
    After switching to pnpm it took 4h to run sonar-scanner
    Before npm: it tooks around 10m

Eventually, I added the following exclusions for all the dist, reports, and other folders in sonar.exclusions:

sonar.sources=packages
sonar.exclusions=**/bower_components/**, **/node_modules/**, **/vendor/**, **/dist/**, **/out/**, **/reports/**, **/.pnpm/**, **/.scannerwork/**

I monitored the process, and it’s clear that the indexing step is taking significantly longer. Some files take around 10-30 minutes to index.

2024-11-11T04:55:04.3002175Z 04:55:04.299 INFO: 4725 files indexed...  (last one was packages/docs-website/scripts/examples/generate-docs-website-examples-index.ts)
...
2024-11-11T05:33:44.3413718Z 05:33:44.340 INFO: 4725 files indexed...  (last one was packages/docs-website/scripts/examples/generate-docs-website-examples-index.ts)

06:51:45.098 INFO: EXECUTION SUCCESS
06:51:45.099 INFO: ------------------------------------------------------------------------
06:51:45.099 INFO: Total time: 3:58:58.809s
06:51:45.199 INFO: Final Memory: 350M/900M

Hey there.

  • What version of SonarQube are you using (you can find this in the footer of your instance)
  • Is the scanner indexing the same number of files as before you made the switch?
  • Probably, the most helpful logs would be DEBUG level scanner logs (sonar-scanner -X) from both before and after you made the switch to pnpm.

Hi Colin

  • SonarQube server 9.9.3.79811
  • The number files is exactly same before and after switching
  • I tried running sonar-scanner -X, but it prompted a lot of logs, and I only noticed that the message 4725 files indexed... was different before and after

Well hopefully the folks over here can look at the logs with a critical eye and try to spot some differences :mag_right:

But without them, I don’t see how anybody can be helpful.

Otherwise, if you can build a small reproducer project that reproduces the issue (longer scan time when using pnpm), maybe that will be helpful.

Hi, I just noticed that removing the node_modules folder in each local project within the monorepo significantly speeds up the indexing process. Eventually, I have already added sonar.exclusions=**/node_modules/** to the configuration before but dont help much until remove the node_modules

I have a few questions:

  • Does the SonarScanner need to interact with node_modules in any way?
  • Can I safely remove it?
  • Will removing it cause any issues?

My expectation is that if you’re excluding the node_modules folder with sonar.exclusions, it should behave as if that folder doesn’t exist.

Once again, I think DEBUG level analysis logs would help here.

Probably, the most helpful logs would be DEBUG level scanner logs (sonar-scanner -X) from both before and after you made the switch to pnpm.