Sonar/scan running in parallel failed with pnpm and monorepo

SonarQube Cloud
,
1
  • Env.
  • Pipeline 
    •     - step: &sonarCloudAnalyze
        name: Sonar affected artefacts
        size: 2x
        artifacts:
          download: true
          paths:
            - "**/sonarcloud-scan.log"
        caches:
          - pnpm
          - sonar
        script:
          - export PATH=./node_modules/.bin:$PATH
          - export PNPM_STORE_PATH="$HOME/.pnpm-store"
          - export BITBUCKET_PIPE_STORAGE_DIR="$BITBUCKET_CLONE_DIR/.bitbucket/pipelines/generated/pipeline/pipes/sonarsource/sonarcloud-scan"
          - echo "Installing Pnpm at global scope..."
          - npm install -g pnpm@latest-10
          - pnpm config set store-dir "$PNPM_STORE_PATH"
          - pnpm install
          - if [ "$BITBUCKET_BRANCH" = "main" ]; then BASE="...[HEAD~1]"; else BASE="...[origin/main]"; fi
          - echo "Using base commit $BASE"
          - >-
            pnpm --filter "$BASE"
            --changed-files-ignore-pattern "**/*.stories.ts"
            --changed-files-ignore-pattern "**/*.md"
            --changed-files-ignore-pattern "./bitbucket-pipelines.yml"
            run sonar

With that configuration, we were able to run sonar scan and we see that the projects are processed in serie, one after the other. After an upgrade to sonar scan 4.3.2 and PNPM v10.17.1 the code below did not work anymore because the PNPM process now the projects in parallel. We can see that at the log. Because of this behavior change, we noticed that the the sonar/can does not support parallelism. We got many checksum error in downloading the JRE or other artefact.

apps/hotels-spa sonar: [ERROR] Bootstrapper: An error occurred: Error: Checksum verification failed for /root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz. Expected checksum bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707 but got aba593a7d0b3f8a0b5a15526984bc6474869affefe62f58c034fd7669777ec4e
apps/hotels-spa sonar: /opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/file.js:108
apps/hotels-spa sonar:             throw new Error(`Checksum verification failed for ${filePath}. Expected checksum ${expectedChecksum} but got ${checksum}`);
apps/hotels-spa sonar:                   ^
apps/hotels-spa sonar: Error: Checksum verification failed for /root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz. Expected checksum bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707 but got aba593a7d0b3f8a0b5a15526984bc6474869affefe62f58c034fd7669777ec4e
apps/hotels-spa sonar:     at validateChecksum (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/file.js:108:19)
apps/hotels-spa sonar:     at async getCacheFileLocation (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/file.js:44:13)
apps/hotels-spa sonar:     at async fetchJRE (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/java.js:87:27)
apps/hotels-spa sonar:     at async runScan (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/scan.js:89:20)
apps/hotels-spa sonar:     at async scan (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/scan.js:34:9)
apps/hotels-spa sonar: Node.js v22.20.0
apps/hotels-spa sonar: Failed

or this one

apps/hotels-spa sonar: [ERROR] Bootstrapper: An error occurred: Error: Checksum verification failed for /root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz. Expected checksum bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707 but got d00838c5bdbe03f16b19471a2a7e004b77b6c3c510c516cbbaac727073e25ec6
apps/hotels-spa sonar: /opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/file.js:108
apps/hotels-spa sonar:             throw new Error(`Checksum verification failed for ${filePath}. Expected checksum ${expectedChecksum} but got ${checksum}`);
apps/hotels-spa sonar:                   ^
apps/hotels-spa sonar: Error: Checksum verification failed for /root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz. Expected checksum bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707 but got d00838c5bdbe03f16b19471a2a7e004b77b6c3c510c516cbbaac727073e25ec6
apps/hotels-spa sonar:     at validateChecksum (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/file.js:108:19)
apps/hotels-spa sonar:     at async getCacheFileLocation (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/file.js:44:13)
apps/hotels-spa sonar:     at async fetchJRE (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/java.js:87:27)
apps/hotels-spa sonar:     at async runScan (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/scan.js:89:20)
apps/hotels-spa sonar:     at async scan (/opt/atlassian/pipelines/agent/build/node_modules/.pnpm/@sonar+scan@4.3.2/node_modules/@sonar/scan/src/scan.js:34:9)
apps/hotels-spa sonar: Node.js v22.20.0
apps/hotels-spa sonar: Failed
/opt/atlassian/pipelines/agent/build/apps/hotels-spa:
 ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  hotels-spa@1.0.0 sonar: `node ../../scripts/ci/sonar-analyze-app.js hotels-spa`

We are able to restore by adding --parallel --workspace-concurrency=1

We also notice that the configuration are note taken in consideration for the flag “sonar.scanner.skipJreProvisioning”: true neither for “sonar.scanner.javaExePath”: “/usr/lib/jvm/java-17-openjdk-amd64/bin/java”

Even if we provide the JRE with the code below in pipeline our JRE is completely ignored by the scanner.

- apt-get update && apt-get install -y openjdk-17-jre-headless
- export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
- export PATH="$JAVA_HOME/bin:$PATH"


const sonarConfig = {
  token: process.env.SONAR_TOKEN,
  options: {
    "sonar.projectKey": project.sonarKey,
    "sonar.organization": "xxxxxxxx",
    "sonar.sources": "src",
    "sonar.tests": "tests/unit",
    "sonar.exclusions":
      "**/tests/*.js,**/src/assets/images/*,**/src/assets/fonts/*,**/src/main.ts,./dist/**,**/node_modules/**,**/.pnpm/**",
    "sonar.language": "js",
    "sonar.typescript.lcov.reportPaths": "coverage/lcov.info",
    "sonar.testExecutionReportPaths": "test-report.xml",
    "sonar.scanner.skipJreProvisioning": true,
    "sonar.scanner.javaExePath": "/usr/lib/jvm/java-17-openjdk-amd64/bin/java"
  },
};


scan(sonarConfig, (err) => {
  console.log = originalConsoleLog;
  logStream.end();

  if (err) {
    console.error(`Sonar scan failed for project ${projectName}:\n${err.stack}`);
    process.exit(1);
  } else {
    console.log(`Scan successful for ${projectName}`);
  }
});

**Question: Is there a way for sonar/scan to be able to process project in parallel (//) as we are under a monorepo ? or could you please make this sonar/scan ready for parallelism in near future ?

And latest tests reports strange error below with the version 4.3.2. Restoring the version 4.3.0 makes it works again.

pnpm --filter "$BASE" --workspace-concurrency=1 --changed-files-ignore-pattern "**/*.stories.ts" --changed-files-ignore-pattern "**/*.md" --changed-files-ignore-pattern "./vitest.config.shared.ts" --changed-files-ignore-pattern "./bitbucket-pipelines.yml" run sonar
1s
+ pnpm --filter "$BASE" --workspace-concurrency=1 --changed-files-ignore-pattern "**/*.stories.ts" --changed-files-ignore-pattern "**/*.md" --changed-files-ignore-pattern "./vitest.config.shared.ts" --changed-files-ignore-pattern "./bitbucket-pipelines.yml" run sonar
Scope: 5 of 6 workspace projects
> hotels-spa@1.0.0 sonar /opt/atlassian/pipelines/agent/build/apps/hotels-spa
> node ../../scripts/ci/sonar-analyze-app.js hotels-spa
Sonar Scan file: /opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes/sonarsource/sonarcloud-scan/sonarcloud-scan.log
Scanning project hotels-spa...
[INFO]  Bootstrapper: Retrieving info from "package.json" file
[INFO]  Bootstrapper: Platform: linux x64
[INFO]  Bootstrapper: Server URL: https://sonarcloud.io
[INFO]  Bootstrapper: Version: 4.3.2
[INFO]  Bootstrapper: JRE provisioning is supported
[INFO]  Bootstrapper: Using JRE from the cache
node:events:497
      throw er; // Unhandled 'error' event
      ^
Error: spawn /root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz_extracted/jdk-17.0.11+9-jre/bin/java ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:285:19)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
Emitted 'error' event on ChildProcess instance at:
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz_extracted/jdk-17.0.11+9-jre/bin/java',
  path: '/root/.sonar/cache/bcb1b7b8ad68c93093f09b591b7cb17161d39891f7d29d33a586f5a328603707/OpenJDK17U-jre_x64_linux_hotspot_17.0.11_9.tar.gz_extracted/jdk-17.0.11+9-jre/bin/java',
  spawnargs: [
    '-jar',
    '/root/.sonar/cache/0006dfb21b1da4e8b32d54b9294a0a58aa6c67789f2692325d0f8ef56c820dbd/sonarcloud-scanner-engine-11.22.0.978.jar'
  ]
}
Node.js v22.20.0
/opt/atlassian/pipelines/agent/build/apps/hotels-spa:
 ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  hotels-spa@1.0.0 sonar: `node ../../scripts/ci/sonar-analyze-app.js hotels-spa`
Exit status 1
4

Hey there.

First question: is your expectation that the scan will run in parallel and submit SonarQube analysis all under 1 project key, or that the scans for multiple SonarQube projects (with distinct keys) are able to run in parallel?