Warn: can't calculate hash

Im still getting a lot of warning with regard to not being able to calculate hash. 23K in fact.

I origianlly started with thread: WARN: Could not calculate hash for class XXXX

16:28:00 [Sonar Analysis for Pull Request] $ /home/ec2-user/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/bin/sonar-scanner -Dsonar.host.url=https://sonarcloud.io ******** -Dsonar.pullrequest.branch=pipeline/kch/sonarqube-test-build -Dsonar.java.skipUnchanged=true -Dsonar.java.source=8 -Dsonar.organization=exerpteam -Dsonar.pullrequest.provider=github -Dsonar.verbose=false "-Dsonar.exclusions=plugin-eft-generated/src/**, plugin-ext-generated/src/**" -Dsonar.projectKey=exerpteam_exerp-platform -Dsonar.java.libraries=./**/*.jar -Dsonar.qualitygate.wait=true -Dsonar.pullrequest.key=3839 -Dsonar.java.binaries=./bootstrap/build/classes,./plugin-localization-impl/build/classes,./plugin-hardware-impl/build/classes,./buildSrc/build/classes,./web-taglib/build/classes,./server-base/build/classes,./test-integration/build/classes,./server-facade-impl/build/classes,./plugin-eft/build/classes,./common-server/build/classes,./test-framework/build/classes,./patch-wildfly-http-client-common/build/classes,./controller-descriptor/build/classes,./plugin-ext/build/classes,./common-module/build/classes,./server-scheduling/build/classes,..... -Dsonar.qualitygate.timeout=300 -Dsonar.pullrequest.github.repository=exerpteam/exerp-platform -Dsonar.pullrequest.base=develop "-Dsonar.projectBaseDir=/home/ec2-user/jenkins/workspace/Sonar Analysis for Pull Request"
16:28:00 INFO: Scanner configuration file: /home/ec2-user/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/conf/sonar-scanner.properties
16:28:00 INFO: Project root configuration file: NONE
16:28:00 INFO: SonarScanner 4.7.0.2747
16:28:00 INFO: Java 17.0.2 Oracle Corporation (64-bit)
16:28:00 INFO: Linux 5.15.122-77.145.amzn2.x86_64 amd64
16:28:00 INFO: SONAR_SCANNER_OPTS=-Xms8G
16:28:01 INFO: User cache: /home/ec2-user/.sonar/cache
16:28:01 INFO: Scanner configuration file: /home/ec2-user/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/conf/sonar-scanner.properties
16:28:01 INFO: Project root configuration file: NONE
16:28:01 INFO: Analyzing on SonarCloud
16:28:01 INFO: Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)
16:28:02 INFO: Load global settings
16:28:02 INFO: Load global settings (done) | time=187ms
16:28:02 INFO: Server id: XXX
16:28:02 INFO: User cache: /home/ec2-user/.sonar/cache
16:28:02 INFO: Loading required plugins
16:28:02 INFO: Load plugins index
16:28:02 INFO: Load plugins index (done) | time=80ms
16:28:02 INFO: Load/download plugins
16:28:02 INFO: Load/download plugins (done) | time=46ms
16:28:03 INFO: Found an active CI vendor: 'Jenkins'
16:28:03 INFO: Load project settings for component key: 'XXX'
16:28:03 INFO: Load project settings for component key: 'XXX' (done) | time=84ms
16:28:03 INFO: Process project properties
16:28:03 INFO: Project key: exerpteam_exerp-platform
16:28:03 INFO: Base dir: /home/ec2-user/jenkins/workspace/Sonar Analysis for Pull Request
16:28:03 INFO: Working dir: /home/ec2-user/jenkins/workspace/Sonar Analysis for Pull Request/.scannerwork
16:28:04 INFO: Load project branches
16:28:05 INFO: Load project branches (done) | time=185ms
16:28:05 INFO: Check ALM binding of project 'exerpteam_exerp-platform'
16:28:05 INFO: Detected project binding: BOUND
16:28:05 INFO: Check ALM binding of project 'exerpteam_exerp-platform' (done) | time=77ms
16:28:05 INFO: Load project pull requests
16:28:05 INFO: Load project pull requests (done) | time=172ms
16:28:05 INFO: Load branch configuration
16:28:06 INFO: Load branch configuration (done) | time=609ms
16:28:06 INFO: Load quality profiles
16:28:06 INFO: Load quality profiles (done) | time=170ms
16:28:06 INFO: Load active rules
16:28:08 INFO: Load active rules (done) | time=2633ms
16:28:09 INFO: Organization key: exerpteam
16:28:09 INFO: Pull request 3839 for merge into develop from pipeline/kch/sonarqube-test-build
16:28:09 INFO: Preprocessing files...
16:28:19 INFO: 
16:28:22 INFO: 1 language detected in 27681 preprocessed files
16:28:22 INFO: 288487 files ignored because of inclusion/exclusion patterns
16:28:22 INFO: 348 files ignored because of scm ignore settings
16:28:22 INFO: Loading plugins for detected languages
16:28:22 INFO: Load/download plugins
16:28:22 INFO: Load/download plugins (done) | time=59ms
16:28:22 INFO: Load project repositories
16:28:27 INFO: Load project repositories (done) | time=4981ms
16:28:27 INFO: SCM collecting changed files in the branch
16:28:27 INFO: SCM collecting changed files in the branch (done) | time=240ms
16:28:27 INFO: Indexing files...
16:28:27 INFO: Project configuration:
16:28:27 INFO:   Included sources: **/*.java
16:28:27 INFO:   Excluded sources: **/build-wrapper-dump.json, plugin-eft-generated/src/**, plugin-ext-generated/src/**
16:28:27 INFO:   Excluded sources for duplication: **/api/**
16:28:28 INFO: 27681 files indexed
16:28:28 INFO: Quality profile for java: Exerp quality profile for Java
16:28:28 INFO: ------------- Run sensors on module exerpteam_exerp-platform
16:28:28 INFO: Load metrics repository
16:28:28 INFO: Load metrics repository (done) | time=71ms
16:28:28 INFO: Sensor cache enabled
16:28:28 INFO: Load sensor cache
16:28:30 INFO: Load sensor cache (73 MB) | time=2227ms
16:28:31 INFO: Sensor JavaSensor [java]
16:28:31 INFO: Configured Java source version (sonar.java.source): 8, preview features enabled (sonar.java.enablePreview): false
16:28:34 INFO: The Java analyzer is running in a context where unchanged files can be skipped. Full analysis is performed for changed files, optimized analysis for unchanged files.
16:28:34 WARN: Could not calculate hash for class com.exerp.clublead.client.base.props.components.SalutationPropertyTypeController

Hi,

Did you follow the advise in the thread you linked to?

 
Ann

Hey, yes I removed the wildcards from sonar.java.binaries.

Hi,

Can you post your full analysis log, please?

 
Thx,
Ann

sonar-scan.log (2.4 MB)

Hi,

Your scanner version has a fair amount of age on it; v4.7 is from Feb 2022 and the current version is 5.0.1.

Can you upgrade and see if that makes a difference?

 
Thx,
Ann

The Jenkins plug is: SonarQube Scanner Version 2.17.2
The sonar scanner has been updated to “SonarScanner 5.0.1.3006”.
But it has changed nothing about the ability to calculate hash.

Hi,

Thanks for checking. I’ve flagged this for more expert eyes.

 
Ann

1 Like

Hello @ShadowNinjaHunter,

Thanks for reaching out. I’ve checked the logs. As far as I understand ypu’re using Gradle. Any specific reason, why you’re using sonarscanner? It is recommended that for Gradle project you’re using Sonar Gradle plugin: Gradle - Plugin: org.sonarqube

From the logs I saw a few warnings like this:
WARN: Invalid character encountered in file /home/ec2-user/jenkins/workspace/Sonar Analysis for Pull Request/eclub2-custom/FITNESSDK/conversion/src/dk/procard/eclub/converter/fitnessdk/FitnessDkSubscriptionFilter.java at line 50 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.

Could you please double-check that file encoding is correct and file endings are appropriate ones?

It would help if you could show the logs with info or debug level.

Best,
Margarita

Hey

I have no clue of why we are running sonarscanner.

File encoding in general is ISO-8859-1 but there are a few that are in UTF-8.
I can try to set the sonar.sourceEncoding=ISO-8859-1

I’ll also do a run with sonar.log.level=debug

Enabling log level debug adds this to the log:

10:20:47.963 DEBUG: keyStore is : 
10:20:47.964 DEBUG: keyStore type is : pkcs12
10:20:47.964 DEBUG: keyStore provider is : 
10:20:47.964 DEBUG: init keystore
10:20:47.965 DEBUG: init keymanager of type SunX509
10:20:48.137 DEBUG: Create: /home/ec2-user/.sonar/cache
10:20:48.139 INFO: User cache: /home/ec2-user/.sonar/cache
10:20:48.139 DEBUG: Create: /home/ec2-user/.sonar/cache/_tmp
10:20:48.141 DEBUG: Extract sonar-scanner-api-batch in temp...
10:20:48.147 DEBUG: Get bootstrap index...
10:20:48.147 DEBUG: Download: https://sonarcloud.io/batch/index
10:20:48.511 DEBUG: Get bootstrap completed
10:20:48.513 DEBUG: Download https://sonarcloud.io/batch/file?name=scanner-sonarcloud-8.0.0.54393-all.jar to /home/ec2-user/.sonar/cache/_tmp/fileCache3155850549267342096.tmp
10:20:49.219 DEBUG: Create isolated classloader...
10:20:49.235 DEBUG: Start temp cleaning...
10:20:49.240 DEBUG: Temp cleaning done
10:20:49.240 INFO: Analyzing on SonarCloud
10:20:49.241 INFO: Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)
10:20:49.241 DEBUG: Work directory: /home/ec2-user/jenkins/workspace/Sonar Analysis for Pull Request/.scannerwork
10:20:49.242 DEBUG: Execution execute

And setting sourceEncoding does not change
WARN: Could not calculate hash for class dk.procard.eclub.base.PropertiesDialogModel

Thanks for the logs; however, I think I need further logs with debug level.

More precisely the logs of the Analyzer. If you can just send me the full log, that will also be fine.

Best,
Margarita

sonar-debug-scan.log (2.7 MB)

I had another look into your logs. So I probably tell you what is happening.

Some analyzers for Java language (Security, DBD) do not just calculate the hash of source files but also the class files to make sure it’s safe to re-use the result from the cache. If it fails, we have to parse files again. In your case, “Could not calculate the hash for class…” appears because the analyzer failed to hash the class file (.class). Most likely it happened because the class file wasn’t found.
To prove this theory you can disable all rules coming from “snarsecurity” and “dbd” repos and check if you still see the message.

Why the class files could be missing? The class files locations are provided in the property sonar.java.binaries as a list of locations. I can see from the logs that you provide them, but I can’t check if you provide all the locations and if the locations are correct.

This is very error-prone to provide this property manually, so we highly recommend to use Scanner for Gradle if you’re building with Gradle, or Scanner for Maven, if you’re using Maven.

So, I recommend you to move to the Scanner for Gradle for the analysis and check if you still have this issue.

Best,
Margarita

1 Like

hey

I have NOT tried to disable “snarsecurity” and “dbd”.

I have hat another look at the sonar.java.binaries and added /java/main to the modules and that has removed the warnings about calculation of hash.

I agree with providing that property manually is every error-prone.

It has also resulted in “Server-side caching is enabled. The Java analyzer was able to leverage cached data from previous analyses for 19812 out of 27686 files.” which reduces the run time to about half. Before it was 0 out of 27686 files.

1 Like

Thanks for the update!

I still recommend you to migrate to the Gradle scanner.

Best,
Margarita

1 Like