CFamily cache limitations?

SonarQube Server / Community Build
1

I am looking for information on the limitations of the caching introduced in SonarCFamily 6.6 (cmp. C++ developers now benefit from more efficient analysis and deeper Core Guidelines coverage)

So far, I’ve only read that it does not support parallel scans.

At https://sonarcloud.io/documentation/analysis/languages/cfamily/#analysis-cache it says “each project should use its own path” - that’s a “should”, but is it really a “must” or would it be okay to use the same analysis cache path for different projects?

What about using the cache for scanning large pull requests (lots of changes). I am talking of at least 2 pull requests that are also being updated, being scanned intertwined (not in parallel), both of which are vastly different, each modifying the master branch’s code extensively (adding files, removing files, etc). Must I create a copy of the master cache prior to scanning the branches? Or is it okay to use the master branch cache for both and are there performance penalties?

What other limits are there? Or maybe: is there a detailed explanation on how the caching works (how are files indexed - name, date, checksum, branch, …?)

I am asking because I am running into the issue that subsequent analyses abort with <number> is not a valid line for pointer. File <filename> has <number less than first number> line(s) (see below). I am not able to properly reproduce it, but it basically prevents me from integrating SonarQube into my CI jobs as it takes almost half an hour to scan that particular project from scratch. Caching would help tremendously, though.

Any help is deeply appreciated, thank you!!

  • SonarQube Scanner 4.2.0.1873 with Java 11.0.3 AdoptOpenJDK (64-bit) on Linux 4.15.0-72-generic amd64
  • SonarQube server 8.1.0 (Developer Edition)
  • SonarCFamily plugin 6.6.0 (build 13759)
  • Other plugins: Git, SonarJS, SonarPython, SonarTS
[...]
ERROR: Error during SonarQube Scanner execution
java.lang.IllegalArgumentException: 76 is not a valid line for pointer. File src/path/to/code.h has 54 line(s)
	at org.sonar.api.utils.Preconditions.checkArgument(Preconditions.java:43)
	at org.sonar.api.batch.fs.internal.DefaultInputFile.checkValid(DefaultInputFile.java:336)
	at org.sonar.api.batch.fs.internal.DefaultInputFile.newPointer(DefaultInputFile.java:272)
	at org.sonar.api.batch.fs.internal.DefaultInputFile.selectLine(DefaultInputFile.java:295)
	at com.sonar.cpp.plugin.CFamilySensor.rangeOrLine(CFamilySensor.java:706)
	at com.sonar.cpp.plugin.CFamilySensor.reportIt(CFamilySensor.java:627)
	at com.sonar.cpp.plugin.CFamilySensor.save(CFamilySensor.java:579)
	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:487)
	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:438)
	at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:277)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:62)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:386)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:382)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:351)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)

code.h is basically #include <string> and definition of a small class.

1 Like
2

Hello @linisgre,

That is a should for the cache to be useful.

The recommended way to benefit from the cache, to have higher hit rate, is to copy the master cache.

Can you walk me through the steps you did to get this error? was it on the first analysis? second one? The setup that led to this? Were you copying the cache of the master? all details are appreciated.

Thanks,
Abbas

3

Can you walk me through the steps you did to get this error? was it on the first analysis? second one? The setup that led to this? Were you copying the cache of the master? all details are appreciated.

As mentioned I am not able to reproduce it. It happened on my Jenkins instance for this one pull request, and I could trigger it once locally. When I wanted to write it down properly, the reproduction steps did not work anymore (probably missed something, or maybe it is indeed not a deterministic occurence).

Assumed sequence:

  1. With empty cache, do a build (build-wrapper… make && sonar-scanner with cache)
  2. Remove sonar-build, keep the cache, run make clean
  3. Repeat step 1 (same code), get the error

There probably are steps missing (dunno which). Maybe it’s something to do with my codebase triggering this, I don’t know.

6

@linisgre,

Since there is no way to reproduce it, I cannot help much.
Please get back to us if it happens again or if you find a way to reproduce it.

7

A post was split to a new topic: Cfamily cache corruption

8

Hi @linisgre,

we identified a bug when analyzing C/C++/Objective-C code and caching enabled, we believe you are affected by it, here is the ticket which will be fixed in the next release: CPP-2458.

9

Hi @linisgre,

FYI the issue you were facing has been fixed in CPP-2458 and has been released as part of SonarCFamily 6.8.

10

A post was split to a new topic: CFamily cache configuration