The "build-wrapper-dump.json" file was found but 0 C/C++/Objective-C files were analyzed - Docker

Szymon · October 9, 2023, 8:56am

ALM used: Azure DevOps
CI system used: Azure DevOps
Languages of the repository: C/C++

Hi,

I am building and testing my project inside Docker container where I am also trying to execute sonar-scanner, based on build wrapper executed at the begining. At the end I was able to find build-wrapper-domp.json but for some reason I’ve got the error like this:

...
2023-10-06T19:08:35.3356790Z 19:08:35.294 INFO: 0/2283 files marked as unchanged
2023-10-06T19:08:35.3356924Z 19:08:35.295 INFO: Cache: 0/0 hits, 125 bytes
2023-10-06T19:08:35.3357059Z 19:08:35.295 INFO: 0 compilation units analyzed
2023-10-06T19:08:35.3357192Z 19:08:35.304 DEBUG: the bridge server will shutdown
2023-10-06T19:08:40.3062200Z 19:08:40.305 DEBUG: the bridge server closed
2023-10-06T19:08:40.3062410Z 19:08:40.305 INFO: Time spent writing ucfgs 227ms
2023-10-06T19:08:40.7969001Z 19:08:40.796 INFO: ------------------------------------------------------------------------
2023-10-06T19:08:40.7969225Z 19:08:40.796 INFO: EXECUTION FAILURE
2023-10-06T19:08:40.7969458Z 19:08:40.796 INFO: ------------------------------------------------------------------------
2023-10-06T19:08:40.7969623Z 19:08:40.796 INFO: Total time: 15:31.980s
2023-10-06T19:08:40.8847742Z 19:08:40.884 ERROR: Error during SonarScanner execution
2023-10-06T19:08:40.8848035Z 19:08:40.884 INFO: Final Memory: 114M/4096M
2023-10-06T19:08:40.8848625Z 19:08:40.884 INFO: ------------------------------------------------------------------------
2023-10-06T19:08:40.8851924Z java.lang.IllegalStateException: The "build-wrapper-dump.json" file was found but 0 C/C++/Objective-C files were analyzed. Please make sure that:
2023-10-06T19:08:40.8852188Z   * you are using the latest version of the build-wrapper and the CFamily analyzer
2023-10-06T19:08:40.8852404Z   * you are correctly invoking the scanner with correct configuration
2023-10-06T19:08:40.8852532Z   * your compiler is supported
2023-10-06T19:08:40.8852653Z   * you are wrapping your build correctly
2023-10-06T19:08:40.8852769Z   * you are wrapping a full/clean build
2023-10-06T19:08:40.8852980Z   * you are providing the path to the correct build-wrapper output directory
2023-10-06T19:08:40.8853159Z   * you are building and analyzing the same source checkout, absolute paths must be identical in build and analysis steps
2023-10-06T19:08:40.8853357Z 	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:611)
2023-10-06T19:08:40.8853536Z 	at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:241)
2023-10-06T19:08:40.8853741Z 	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:62)
2023-10-06T19:08:40.8853950Z 	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
2023-10-06T19:08:40.8854167Z 	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:51)
2023-10-06T19:08:40.8854374Z 	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:64)
2023-10-06T19:08:40.8854590Z 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2023-10-06T19:08:40.8854796Z 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2023-10-06T19:08:40.8855076Z 	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:163)
2023-10-06T19:08:40.8855285Z 	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:159)
2023-10-06T19:08:40.8855503Z 	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:130)
2023-10-06T19:08:40.8855715Z 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2023-10-06T19:08:40.8855925Z 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2023-10-06T19:08:40.8856126Z 	at org.sonar.scanner.bootstrap.ScannerContainer.doAfterStart(ScannerContainer.java:396)
2023-10-06T19:08:40.8856340Z 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2023-10-06T19:08:40.8856544Z 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2023-10-06T19:08:40.8856752Z 	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:128)
2023-10-06T19:08:40.8856962Z 	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
2023-10-06T19:08:40.8857173Z 	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
2023-10-06T19:08:40.8857361Z 	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:57)
2023-10-06T19:08:40.8857530Z 	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:51)
2023-10-06T19:08:40.8857740Z 	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
2023-10-06T19:08:40.8857940Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-10-06T19:08:40.8858231Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-10-06T19:08:40.8858399Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-10-06T19:08:40.8858562Z 	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-10-06T19:08:40.8858748Z 	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
2023-10-06T19:08:40.8858933Z 	at com.sun.proxy.$Proxy0.execute(Unknown Source)
2023-10-06T19:08:40.8859100Z 	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
2023-10-06T19:08:40.8859377Z 	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
2023-10-06T19:08:40.8859557Z 	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
2023-10-06T19:08:40.8859727Z 	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
2023-10-06T19:08:40.8859888Z 	at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Do you have any idea, how to fix this issue?

If you would like to get full build log etc then just let me know, the log file is to big to attach it here

ganncamp · October 10, 2023, 1:12pm

Hi,

Your problem is Docker, and the docs are explicit about it (emphasis mine):

Important notes

The Build Wrapper collects information about the build, including absolute file paths (source files, standard headers, libraries, etc…). Later on, SonarScanner uses this information and needs to access those paths. While this is straightforward when running these two steps on the same host, it is worth considering when using any containerization. A consequence of this is that C/C++/Objective-C analysis is NOT supported by SonarScanner CLI Docker image.

HTH,
Ann

Szymon · October 25, 2023, 9:45am

It seems to me I am trying to do it in a little bit different way. Yes, I’ve got the predefined Docker image and once I am creating the container based on such image I am building my app there and inside used dockerfile I am downloading and unzipping build-wrapper as well as sonar-scanner like this way:

RUN cd /xxx  \
    && curl -sL https://sonarcloud.io/static/cpp/build-wrapper-linux-x86.zip  --output build-wrapper.zip  \
    && unzip build-wrapper.zip
ENV PATH="${PATH}:/xxx/build-wrapper-linux-x86"

RUN cd /xxx  \
    && curl -sL https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.7.0.2747-linux.zip  --output sonar-scanner.zip \
    && unzip sonar-scanner.zip
ENV PATH="${PATH}:/xxx/sonar-scanner-4.7.0.2747-linux/bin"

Together with wrapped (by build wrapper) compilation execution I am also running sonar-scanner -X command so I am not sure If I am using SonarScanner CLI Docker Image. It looks more or less like below:

docker run \
      --name $(Agent.Name) \
      --rm \
      --volume $(pwd):/xxx \
      --env BUILD_PASS \
      --env SONAR_SCANNER_OPTS="-Xmx4G -Xms4G -XX:MaxRAMPercentage=80 -XX:MinRAMPercentage=50" \
      --env SONAR_RUNNER_OPTS="-Xmx4G -Xms4G -XX:MaxRAMPercentage=80 -XX:MinRAMPercentage=50" \
      --env JAVA_OPTS="-Xmx4G -Xms4G -XX:MaxRAMPercentage=80 -XX:MinRAMPercentage=50" \
      --env _JAVA_OPTIONS="-Xmx4G -Xms4G -XX:MaxRAMPercentage=80 -XX:MinRAMPercentage=50" \
      image/xxx /bin/bash -c \
      "./builds/build.sh <build_parameters> && sonar-scanner -X"

Inisde the build.sh script there is wrapped make command like:

build-wrapper-linux-x86-64 --out-dir /home/builder_sc/build-wrapper-linux-x86 make <make_parameters>

AND

everything looks good until analyzing cpp files. Any other extensions like .html, .py, .xml, .js were recognized by the sonar scanner and properly analyzed, but if it comes to cpp files and proper header files then the sonar scanner is able to indexed it but is unable to analyze anything.

I’ve also tried with generate compilation database JSON file by wrapping make command by one of scan-build package command (intercept-build in this case) like below:

intercept-build --verbose --cdb xxx/compile_commands.json --override-compiler make <make_parameters>

Even if I’ve changed it to the compilation database (instead of build-wrapper), I’ve got similar error at the end like:

10:56:09.237 DEBUG: ...
10:56:09.237 DEBUG: /xxx/packages/xxx/ifmgr/ifmgr_dhcp.h not marked as unchanged: file added
10:56:09.237 DEBUG: /xxx/packages/xxx/lted/Daemon/test/SerialStateMachineTest.cpp not marked as unchanged: file added
10:56:09.237 INFO: 0/2283 files marked as unchanged
10:56:09.238 INFO: Cache: 0/0 hits, 125 bytes
10:56:09.238 INFO: 0 compilation units analyzed
10:56:09.247 DEBUG: the bridge server will shutdown
10:56:14.248 DEBUG: the bridge server closed
10:56:14.248 INFO: Time spent writing ucfgs 255ms
10:56:14.856 INFO: ------------------------------------------------------------------------
10:56:14.856 INFO: EXECUTION FAILURE
10:56:14.856 INFO: ------------------------------------------------------------------------
10:56:14.856 INFO: Total time: 2:32.470s
10:56:14.952 ERROR: Error during SonarScanner execution
10:56:14.952 INFO: Final Memory: 110M/4096M
10:56:14.952 INFO: ------------------------------------------------------------------------
java.lang.IllegalStateException: The Compilation Database JSON file was found but 0 C/C++/Objective-C files were analyzed. Please make sure that:
  * you are correctly invoking the scanner with correct configuration
  * your compiler is supported
  * you are providing the path to the correct Compilation Database JSON
  * you are building and analyzing the same source checkout, absolute paths must be identical in build and analysis steps
	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:611)
	at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:241)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:62)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:51)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:64)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:163)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:159)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:130)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.bootstrap.ScannerContainer.doAfterStart(ScannerContainer.java:396)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:128)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:57)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:51)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Additionally, I’ve checked generated compilation database JSON file and it looks more or less good. Some slice from such file:

[
  {
        "arguments": [
            "cc",
            "-c",
            "-DHAVE_CONFIG_H",
            "-O2",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/staging_dir/host/include",
            "-I.",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0/libiberty/../include",
            "-W",
            "-Wall",
            "-Wwrite-strings",
            "-Wc++-compat",
            "-Wstrict-prototypes",
            "-Wshadow=local",
            "-pedantic",
            "-D_GNU_SOURCE",
            "-o",
            "alloca.o",
            "../../gcc-7.5.0/libiberty/alloca.c"
        ],
        "directory": "/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0-initial/libiberty",
        "file": "../../gcc-7.5.0/libiberty/alloca.c"
  },
  {
        "arguments": [
            "cc",
            "-c",
            "-DHAVE_CONFIG_H",
            "-O2",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/staging_dir/host/include",
            "-I.",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0/libiberty/../include",
            "-W",
            "-Wall",
            "-Wwrite-strings",
            "-Wc++-compat",
            "-Wstrict-prototypes",
            "-Wshadow=local",
            "-pedantic",
            "-D_GNU_SOURCE",
            "-fpic",
            "-o",
            "pic/rust-demangle.o",
            "../../gcc-7.5.0/libiberty/rust-demangle.c"
        ],
        "directory": "/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0-minimal/libiberty",
        "file": "../../gcc-7.5.0/libiberty/rust-demangle.c"
  },
  {
        "arguments": [
            "cc",
            "-c",
            "-DHAVE_CONFIG_H",
            "-I.",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0/libiberty/../include",
            "-W",
            "-Wall",
            "-Wwrite-strings",
            "-Wc++-compat",
            "-Wstrict-prototypes",
            "-Wshadow=local",
            "-pedantic",
            "-D_GNU_SOURCE",
            "-o",
            "hex.o",
            "../../../gcc-7.5.0/libiberty/hex.c"
        ],
        "directory": "/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0-initial/build-x86_64-pc-linux-gnu/libiberty",
        "file": "../../../gcc-7.5.0/libiberty/hex.c"
  },
  {
        "arguments": [
            "cc",
            "-c",
            "-DHAVE_CONFIG_H",
            "-O2",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/staging_dir/host/include",
            "-I.",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0/libiberty/../include",
            "-W",
            "-Wall",
            "-Wwrite-strings",
            "-Wc++-compat",
            "-Wstrict-prototypes",
            "-Wshadow=local",
            "-pedantic",
            "-D_GNU_SOURCE",
            "-fpic",
            "-o",
            "pic/regex.o",
            "../../gcc-7.5.0/libiberty/regex.c"
        ],
        "directory": "/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0-final/libiberty",
        "file": "../../gcc-7.5.0/libiberty/regex.c"
  },
  {
        "arguments": [
            "cc",
            "-c",
            "-DHAVE_CONFIG_H",
            "-I.",
            "-I/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0/libiberty/../include",
            "-W",
            "-Wall",
            "-Wwrite-strings",
            "-Wc++-compat",
            "-Wstrict-prototypes",
            "-Wshadow=local",
            "-pedantic",
            "-D_GNU_SOURCE",
            "-o",
            "sha1.o",
            "../../../gcc-7.5.0/libiberty/sha1.c"
        ],
        "directory": "/xxx/thirdparty/xxx/openwrt.org/openwrt/build_dir/toolchain-aarch64_generic_gcc-7.5.0_glibc/gcc-7.5.0-initial/build-x86_64-pc-linux-gnu/libiberty",
        "file": "../../../gcc-7.5.0/libiberty/sha1.c"
  }
]

Any ideas?? Looks like I am not really far away form solve my case

ganncamp · October 25, 2023, 12:13pm

Hi,

Are you absolutely sure this is the case?

Ann

Szymon · October 25, 2023, 12:26pm

Yes, I am sure. Under each “xxx” they are always the same slice of path.

michael.jabbour · October 25, 2023, 2:29pm

Hi @Szymon, and thanks for sharing the issue with us.

It seems that you have already generated the verbose scanner logs. It would be really nice if you could attach them for us somehow in addition to the files from the build-wrapper output directory (log + json dump).

This should make it much easier for us to identify the problem…

Best regards,
Michael

Szymon · October 25, 2023, 7:18pm

Thanks @michael.jabbour, could I sent that files to you personally?

Szymon · October 26, 2023, 7:36am

@michael.jabbour I’ve already send you all files privately. Thanks in advance!

Szymon · November 3, 2023, 8:21am

Looks like during the compilation the source code files are copied out to the specific file structure (required by compilation) and with “new paths” inside build-wrapper-dump.json file it doesn’t work. Once I’ve replaced “new paths” with old ones (just after checkout them from the repo) inside build-wrapper-dump.json it works properly!

@michael.jabbour thanks for you investigation!

michael.jabbour · November 3, 2023, 1:27pm

Hi @Szymon, thanks for letting us know and I am happy to know that you got the analysis working!

One thing worth noting here is that manipulating the build-wrapper-dump.json can be problematic, and we always recommend users analyze in the same environment they are building using the same file structure used during the build. To mention a few reasons:

The build-wrapper-dump.json format can change at any time. This can break any scripts that you might be using to replace the paths collected there before starting the analysis.
The analyzer may need to access libraries that are installed on the system if they are used during the build. This is necessary for example when analyzing a file that #includes such libraries.
The analyzer may need to probe the compiler being used to mimic their behavior when analyzing a file.
The “0 C/C++/Objective-C files were analyzed error” is resolved as soon as a single file is matched. It might be a tough task to look at the logs and make sure that all files from the build are analyzed correctly.
If I understand correctly, in your case you have to copy source files to achieve a certain file structure during the build. Using the same information captured during the build to analyze files in their source file structure can break relative include paths and reduce the quality of the analysis.

Therefore, I would strongly recommend avoiding the need for copying the files and changing the structure during the build. We are also happy to receive any feedback on why it might be hard to change the build script on your side, and if there is something that you think we might want to consider adding support for about this.

Best regards,
Michael

system · November 10, 2023, 1:27pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.