Null pointer exception with using cache in C++ project

SonarQube Cloud
1

We’re seeing a crash when using a cache with our C++ project.

Our steps for analysis look like this:

  • Sync down cache from S3 (blocked by a lock in Jenkins)
  • Run sonar scanner command with cache enabled
  • Sync up cache to S3 (blocked by a lock in Jenkins)

Project info:

  • ALM used: GitHub
  • CI system used: Self-hosted Jenkins
  • Scanner command used when applicable (private details masked): /home/jenkins/.sonar/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner -Dsonar.login=**** -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=swift-nav -Dsonar.projectName=starling -Dsonar.projectKey=swift-nav_starling -Dsonar.pullrequest.base=master -Dsonar.pullrequest.branch=use_project_specific_sonar_cache -Dsonar.pullrequest.key=7097 -Dsonar.coverageReportPaths=./build/code_coverage.xml -Dproject.settings=build/sonar-project.properties -Dsonar.cfamily.compile-commands=./build/compile_commands.json -Dsonar.cfamily.threads=8 -Dsonar.cfamily.cache.path=/home/jenkins/sonar-scanner-cache/starling -Dsonar.cfamily.cache.enabled=true
  • Languages of the repository: C++
  • Error observed:
[2022-10-28T13:19:57.755Z] INFO: ------------------------------------------------------------------------
[2022-10-28T13:19:57.755Z] INFO: EXECUTION FAILURE
[2022-10-28T13:19:57.755Z] INFO: ------------------------------------------------------------------------
[2022-10-28T13:19:57.755Z] INFO: Total time: 2:55.771s
[2022-10-28T13:19:57.755Z] INFO: Final Memory: 52M/187M
[2022-10-28T13:19:57.755Z] INFO: ------------------------------------------------------------------------
[2022-10-28T13:19:57.755Z] ERROR: Error during SonarScanner execution
[2022-10-28T13:19:57.755Z] java.lang.NullPointerException
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.canonicalizeFilename(CFamilySensor.java:1188)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.lambda$save$20(CFamilySensor.java:1065)
[2022-10-28T13:19:57.755Z] 	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Unknown Source)
[2022-10-28T13:19:57.755Z] 	at java.base/java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.save(CFamilySensor.java:1065)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:1006)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.lambda$process$17(CFamilySensor.java:895)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CompileCommandsReader.readCaptures(CompileCommandsReader.java:61)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:870)
[2022-10-28T13:19:57.755Z] 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:400)
2

We’re also seeing this error occasionally:

[2022-11-02T19:47:26.866Z] ERROR: Error during SonarScanner execution
[2022-11-02T19:47:26.866Z] java.lang.IllegalStateException: java.nio.file.NoSuchFileException: /home/jenkins/sonar-scanner-cache/starling/analysis_state/pvt_common/include/pvt_common/functional/fold.h
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.cache.FilesystemCacheReader.read(FilesystemCacheReader.java:29)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.cache.CacheInImpl.readExternalIssuesCacheName(CacheInImpl.java:208)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.cache.CacheInImpl.query(CacheInImpl.java:88)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:1001)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.CFamilySensor.lambda$process$17(CFamilySensor.java:895)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.CompileCommandsReader.readCaptures(CompileCommandsReader.java:61)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:870)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:400)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:212)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:62)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:51)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:64)
[2022-11-02T19:47:26.866Z] 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
[2022-11-02T19:47:26.866Z] 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:468)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:464)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:420)
[2022-11-02T19:47:26.866Z] 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
[2022-11-02T19:47:26.866Z] 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
[2022-11-02T19:47:26.866Z] 	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:130)
[2022-11-02T19:47:26.866Z] 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
[2022-11-02T19:47:26.866Z] 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
[2022-11-02T19:47:26.866Z] 	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:58)
[2022-11-02T19:47:26.866Z] 	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:52)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
[2022-11-02T19:47:26.866Z] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2022-11-02T19:47:26.866Z] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
[2022-11-02T19:47:26.866Z] 	at com.sun.proxy.$Proxy0.execute(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
[2022-11-02T19:47:26.866Z] 	at org.sonarsource.scanner.cli.Main.main(Main.java:61)
[2022-11-02T19:47:26.866Z] Caused by: java.nio.file.NoSuchFileException: /home/jenkins/sonar-scanner-cache/starling/analysis_state/pvt_common/include/pvt_common/functional/fold.h
[2022-11-02T19:47:26.866Z] 	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/java.nio.file.Files.newByteChannel(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/java.nio.file.Files.newByteChannel(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at java.base/java.nio.file.Files.newInputStream(Unknown Source)
[2022-11-02T19:47:26.866Z] 	at com.sonar.cpp.plugin.cache.FilesystemCacheReader.read(FilesystemCacheReader.java:27)
5

Hi @silverjam,

This error is clear. It means that this file was accessed during the build but wasn’t found during the analysis.

Are you making sure you have the same build and analysis enviroment?

Can you share the full verbose log of this when it happens?

Thanks,

6

Hi Abbas Sabra.

/home/jenkins/sonar-scanner-cache/starling/analysis_state/ path is path to Sonar cache and is created after the build.

8

Hi @rafalparadowski and @silverjam ,

I am going to follow up on the topic. By looking at both errors I suspect that the cache is corrupted and that you may have got multiple writers overriding part of the cache. Would you be able to share how you are storing the cache on s3 to see if I can spot something strange? If you prefer to not share publicly I can open a private topic.

9

Our current setup isolates the cache per project, but multiple jobs may be reading from the cache simultaneously if they are running on the same Jenkins build node. Build and analysis environments are done in the same Jenkins pipeline and on the same node/environment as far as I know.

10

Our S3 sync code looks like this:

/**
 * runWithS3Sync - sync from and to s3 at the beginning and end of the action.
 * @param bucketName: name of s3 bucket
 * @param directory:  path to directory that will be synced with s3
 */
def call(String bucketName, String directory, Closure operation=null) {
    lockName = bucketName + "_sync"
    lock(resource: lockName) {
        sh("aws s3 sync s3://${bucketName} ${directory}")
    }
    operation()
    lock(resource: lockName, skipIfLocked: true) {
        sh("aws s3 sync ${directory} s3://${bucketName}")
    }
}
11

Hi @silverjam ,

this is not supported, you should make sure that each analysis is running with its own cache in isolation.

a tip about cache storage; it might be faster to always pack and unpack the folder rather than running sync, with something like below:

aws s3 cp s3://${bucketName}/${cacheName}.tgz .
tar xvzf ${cacheName}.tgz

// run analysis

tar cvzf ${cacheName}.tgz ${directory}
aws s3 cp ${cacheName}.tgz s3://${bucketName}
12

Hi @rafalparadowski @silverjam ,

did you manage to solve the issue?

13

No, we tried to isolate caches per build so that nothing is sharing a cache, but it did not seem to resolve the issue.

14

Hi @silverjam ,

I would suggest isolating the cache and cleaning all the current caches and starting from scratch.

1 Like