SonarScanner failed on CI environment with error: INVALID HASH

Wasenshi123 · June 10, 2024, 6:17pm

Must-share information (formatted with Markdown):

which versions are you using:
- SonarQube Community Edition v10.5.1,
- SonarScanner for MSBuild 6.2 (from dotnet global tool)
how is SonarQube deployed: Helm
what are you trying to achieve:
Trying to integrate sonarqube analysis to CI. Tried both Github Action & CircleCi.
what have you tried so far to achieve this:
My project is .NET 6, so I use dotnet global tool for scanner.

I always get the error like this on both CI platforms:

INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.11 Azul Systems, Inc. (64-bit)
INFO: Linux 6.5.0-1021-azure amd64
INFO: User cache: /root/.sonar/cache
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 22.551s
INFO: Final Memory: 5M/24M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
org.sonarsource.scanner.api.internal.ScannerException: Unable to execute SonarScanner analysis
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:85)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:74)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:70)
	at org.sonarsource.scanner.api.EmbeddedScanner.doStart(EmbeddedScanner.java:185)
	at org.sonarsource.scanner.api.EmbeddedScanner.start(EmbeddedScanner.java:123)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:74)
	at org.sonarsource.scanner.cli.Main.main(Main.java:62)
Caused by: java.lang.IllegalStateException: INVALID HASH: File /root/.sonar/cache/_tmp/fileCache6157831180582714998.tmp was expected to have hash 72774e07cc9cba708dccb7fc4f4c451e but was downloaded with hash 612e61d79954d84cf5468e3044e43f7a
	at org.sonarsource.scanner.api.internal.cache.FileCache.get(FileCache.java:87)
	at org.sonarsource.scanner.api.internal.JarDownloader.lambda$getScannerEngineFiles$0(JarDownloader.java:60)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.sonarsource.scanner.api.internal.JarDownloader.getScannerEngineFiles(JarDownloader.java:61)
	at org.sonarsource.scanner.api.internal.JarDownloader.download(JarDownloader.java:53)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:76)
	... 7 more

I tried to use sonar.verbose=true, and found this:

04:16:52.675 INFO: SonarScanner 5.0.1.3006
04:16:52.675 INFO: Java 17.0.11 Alpine (64-bit)
04:16:52.675 INFO: Linux 5.15.0-1057-aws amd64
04:16:52.810 DEBUG: keyStore is : 
04:16:52.827 DEBUG: keyStore type is : pkcs12
04:16:52.827 DEBUG: keyStore provider is : 
04:16:52.827 DEBUG: init keystore
04:16:52.827 DEBUG: init keymanager of type SunX509
04:16:52.930 DEBUG: Create: /root/.sonar/cache
04:16:52.931 INFO: User cache: /root/.sonar/cache
04:16:52.931 DEBUG: Create: /root/.sonar/cache/_tmp
04:16:52.933 DEBUG: Extract sonar-scanner-api-batch in temp...
04:16:52.935 DEBUG: Get bootstrap index...
04:16:52.935 DEBUG: Download: https://sonarqube.[mywebsite].com/batch/index
04:16:54.122 DEBUG: Get bootstrap completed
04:16:54.125 DEBUG: Download https://sonarqube.[mywebsite].com/batch/file?name=sonar-scanner-engine-shaded-10.5.1.90531-all.jar to /root/.sonar/cache/_tmp/fileCache2401177890927933551.tmp
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
....
Caused by: java.lang.IllegalStateException: INVALID HASH: File /root/.sonar/cache/_tmp/fileCache2401177890927933551.tmp was expected to have hash 72774e07cc9cba708dccb7fc4f4c451e but was downloaded with hash 23ca861e0e6f732c15b9c611258f9c36
....

When I run the analysis step manually on my local machine I can run them just fine.

What would be the cause of this? How can I investigate it further and make it work on CI environment?

ganncamp · June 11, 2024, 12:23pm

Hi,

My guess is that there’s something “helpful” on your network munging the file in the download. You should probably talk to your network folks.

HTH,
Ann

Wasenshi123 · June 12, 2024, 3:16pm

I further investigated about the network issue

I try POC by using my ubuntu, local machine, to download the problematic file manually from my sonarqube server. (Since the image that I instructed both CI platforms to use for runner/build agent were basically linux’s ubuntu)

when I use

wget https://sonarqube.[mywebsite].com/batch/file?name=sonar-scanner-engine-shaded-10.5.1.90531-all.jar -o test

then check its checksum, I get invalid checksum, the same behavior just like in the CI environment!

If I use

curl https://sonarqube.[mywebsite].com/batch/file?name=sonar-scanner-engine-shaded-10.5.1.90531-all.jar -o test

and checksum the file, then I get correct checksum!

What’s the meaning of this? could this relate to the issue?

ganncamp · June 12, 2024, 3:56pm

Hi,

Since there is a way for you to get the correct checksum, that tells me the file on the server is good.

I’m going to blame your network again. You likely have a proxy that’s interfering with the requests and they just forgot to ~~mangle~~ handle curl requests.

Ann

Wasenshi123 · June 12, 2024, 4:08pm

For clarity, you mean at my sonarqube server network, or at CI network?

ganncamp · June 12, 2024, 4:22pm

Hi,

Uhm… the same proxy wouldn’t be in front of both?

Anyway, if it’s two different ones, I suppose start with the proxy in front of SonarQube.

HTH,
Ann

Wasenshi123 · June 13, 2024, 6:44am

Weird thing is, if it try to download manually with CI (instruct Github Action to wget ${{ secrets.SONAR_HOST_URL }}/batch/file?name=sonar-scanner-engine-shaded-10.5.1.90531-all.jar -O test.tmp or curl ${{ secrets.SONAR_HOST_URL }}/batch/file?name=sonar-scanner-engine-shaded-10.5.1.90531-all.jar -O test.tmp) then checksum the file, it can download successfully.

Only via sonarscanner end step, it failed to download?

Wasenshi123 · June 13, 2024, 7:07am

Do you think it could relate to how underlying downloading logic in java work? (Since with dotnet sonarscanner, I need to manually install java sdk on the build agent. And I’m not java expert, perhaps I could installed a wrong type or version of sdk? perhaps each OS should use different version of sdk?)

ganncamp · June 13, 2024, 11:24am

Hi,

No.

Have you talked to your network folks?

Ann

Wasenshi123 · June 13, 2024, 2:04pm

yes, I tried to communicate with them. They confirmed there are packet drops. But they only blame that is because firewall and I need to tell them what port(s) I need. But I believe sonarscanner and sonarqube only use default https port? So I’m not sure what to do or ask them further.

ganncamp · June 13, 2024, 2:13pm

Hi,

Yes, it’s standard web traffic.

Ehm… I’m certainly no expert, but it sounds like a minimization/deflection of the real problem.

Could they maybe just let SonarQube and the build agents talk directly to each other without the packet-dropping firewall in the way?

Ann

Wasenshi123 · June 13, 2024, 2:24pm

Updated news: I have verified and found a valid way to make CI’s build agent to runs successfully. (But my problem still persists, continue to read on)

Before, up until now, I tried to use build agent running an Ubuntu. But latest test run, I changed to Windows runner, and then it worked.

Working setup:

jobs:
  build:
    name: Build and analyze
    runs-on: windows-latest
    
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis
      - name: Set up JDK 17
        uses: actions/setup-java@v2
        with:
          java-version: 17
          distribution: 'microsoft'
      - name: Cache SonarQube packages
        uses: actions/cache@v1
        with:
          path: ~\.sonar\cache
          key: ${{ runner.os }}-sonar
          restore-keys: ${{ runner.os }}-sonar
      - name: Cache SonarQube scanner
        id: cache-sonar-scanner
        uses: actions/cache@v1
        with:
          path: .\.sonar\scanner
          key: ${{ runner.os }}-sonar-scanner-v3
          restore-keys: ${{ runner.os }}-sonar-scanner-v3
      - name: Install SonarQube scanner
        if: steps.cache-sonar-scanner.outputs.cache-hit != 'true'
        shell: powershell
        run: |
          New-Item -Path .\.sonar\scanner -ItemType Directory
          dotnet tool update dotnet-sonarscanner --tool-path .\.sonar\scanner
      - name: Build and analyze
        shell: powershell
        run: |
          .\.sonar\scanner\dotnet-sonarscanner begin /k:"poc" /d:sonar.token="${{ secrets.SONAR_TOKEN }}" /d:sonar.host.url="${{ secrets.SONAR_HOST_URL }}" /d:sonar.cs.opencover.reportsPaths="**/TestResults/**/coverage.opencover.xml" -d:sonar.cs.vstest.reportsPaths="**/TestResults/*.trx" /d:sonar.verbose=true
          dotnet test ./SonarPOC/SonarPOC.sln --logger trx --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover
          .\.sonar\scanner\dotnet-sonarscanner end /d:sonar.token="${{ secrets.SONAR_TOKEN }}"

But I still have problem, because I needs to run with Ubuntu so that I can create a side container on CI’s build agent (a postgres db) which is required to make my Test suite all green pass.

(Windows runner is not allowed to create side container in Github action)

My question is, does this result suggest that for .NET I need to use windows runner to make sonarscanner work? Or what other possibility? Because now it seems not to be the self-hosted sonarqube network issue.

Topic		Replies	Views
SonarQube fails with Invalid hash SonarQube Server / Community Build github-actions	11	508	December 14, 2024
New sonar project - INVALID HASH exception SonarQube Server / Community Build	0	1750	June 26, 2018
Error During SonarScanner CLI Execution: Hash Mismatch SonarQube Server / Community Build sonarqube , scanner	1	79	November 4, 2024
Fail to download sonar-scanner-engine-shaded-9.7.1.62043-all.jar SonarQube Server / Community Build sonarqube , scanner	7	2220	March 21, 2024
SonarQube runner failes with mismatched plugin hash SonarQube Server / Community Build	5	2214	November 7, 2019

SonarScanner failed on CI environment with error: INVALID HASH

Related topics