Sonarqube Problems after Upgrade 8.0

Hello,

we upgrade from Sonarqube 7.9.1 and Sonarscanner 4.0.0.1744 to Sonarqube 8.0 and Sonarscanner 4.2.0.1873 and Bamboo 6.10.3.

Ater the Upgrade we have a Problem with analyze and check of a Project building use gcc and xsdk from Xilinx Centos 7 Docker.

Before

|build|07-Okt-2019 16:02:55|14:02:55.687 DEBUG: 'sla_top.sdk/R5_HiL_Replay_App/src/tx_dma.c' generated metadata with charset 'UTF-8'|
|---|---|---|
|build|07-Okt-2019 16:02:55|14:02:55.705 INFO: 0 compilation units analyzed|
|build|07-Okt-2019 16:02:55|14:02:55.705 INFO: Sensor CFamily [cpp] (done) | time=1302ms|
|build|07-Okt-2019 16:02:55|14:02:55.705 INFO: Sensor JavaSecuritySensor [security]|
|build|07-Okt-2019 16:02:55|14:02:55.705 INFO: Reading type hierarchy from: /opt/bamboo-server-home/xml-data/build-dir/4554753/SLA-R5FIR163-BUIL/.scannerwork/ucfg2/java|
|build|07-Okt-2019 16:02:55|14:02:55.705 INFO: Read 0 type definitions|
|build|07-Okt-2019 16:02:55|14:02:55.707 INFO: Reading UCFGs from: /opt/bamboo-server-home/xml-data/build-dir/4554753/SLA-R5FIR163-BUIL/.scannerwork/ucfg2/java|
|build|07-Okt-2019 16:02:55|14:02:55.708 INFO: No UCFGs have been included for analysis.|

work_plan-14745773-BUIL-5.txt (354.9 KB)

After:
build 05-Nov-2019 11:37:30 10:37:30.370 INFO: 0 compilation units analyzed
build 05-Nov-2019 11:37:30 10:37:30.390 INFO: ------------------------------------------------------------------------
build 05-Nov-2019 11:37:30 10:37:30.390 INFO: EXECUTION FAILURE
build 05-Nov-2019 11:37:30 10:37:30.390 INFO: ------------------------------------------------------------------------
build 05-Nov-2019 11:37:30 10:37:30.391 INFO: Total time: 15.386s
build 05-Nov-2019 11:37:30 10:37:30.460 INFO: Final Memory: 22M/160M
error 05-Nov-2019 11:37:30 10:37:30.460 ERROR: Error during SonarQube Scanner execution
build 05-Nov-2019 11:37:30 10:37:30.460 INFO: ------------------------------------------------------------------------
error 05-Nov-2019 11:37:30 java.lang.IllegalStateException: The “build-wrapper-dump.json” file was found but 0 C/C++/Objective-C files were analyzed. Please make sure that:
error 05-Nov-2019 11:37:30 * you are using the latest version of the build-wrapper and the SonarCFamily analyzer
error 05-Nov-2019 11:37:30 * you are correctly invoking the scanner with correct configuration * your compiler is supported
error 05-Nov-2019 11:37:30 * you are wrapping your build correctly
error 05-Nov-2019 11:37:30 * you are wrapping a full/clean build
error 05-Nov-2019 11:37:30 * you are providing the path to the correct build-wrapper output directory
error 05-Nov-2019 11:37:30 at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:247)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:62)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)
error 05-Nov-2019 11:37:30 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
error 05-Nov-2019 11:37:30 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:387)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:383)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:346)
error 05-Nov-2019 11:37:30 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
error 05-Nov-2019 11:37:30 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
error 05-Nov-2019 11:37:30 at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)
error 05-Nov-2019 11:37:30 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
error 05-Nov-2019 11:37:30 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
error 05-Nov-2019 11:37:30 at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
error 05-Nov-2019 11:37:30 at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
error 05-Nov-2019 11:37:30 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
error 05-Nov-2019 11:37:30 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
error 05-Nov-2019 11:37:30 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
error 05-Nov-2019 11:37:30 at java.base/java.lang.reflect.Method.invoke(Unknown Source)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
error 05-Nov-2019 11:37:30 at com.sun.proxy.$Proxy0.execute(Unknown Source)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
error 05-Nov-2019 11:37:30 at org.sonarsource.scanner.cli.Main.main(Main.java:61)
error 05-Nov-2019 11:37:30 10:37:30.462 ERROR:
error 05-Nov-2019 11:37:30 10:37:30.462 ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
simple 05-Nov-2019 11:37:30 Failing task since return code of [/data/bamboo-server-install/atlassian-bamboo-6.10.3/temp/runInDocker6428110199497244648.sh /opt/sonar-scanner-4.2.0.1873-linux/bin/sonar-scanner -Dsonar.projectKey=FE_SLA_r5-replay -Dsonar.sources=sla_top.sdk/R5_HiL_Replay_App/src -Dsonar.cfamily.build-wrapper-output=build_output -Dsonar.log.level=DEBUG -Dsonar.cfamily.gcov.reportsPath=sla_top.sdk/R5_HiL_Replay_App -Dsonar.branch.name=bugfix/SLA-4274-replay-info-in-logging-mode -Dsonar.host.url=http://10.10.67.30:9000 -Dsonar.login=****** was 1 while expected 0
simple 05-Nov-2019 11:37:30 SONAR4BAMBOO: was not able to find a SonarQube result URL
simple 05-Nov-2019 11:37:30 Finished task ‘Analyze code and upload sonarqube result’ with result: Failedfailed_plan-14745773-BUIL-6.txt (347.2 KB)

On Bamboo we have this parameters:
-Dsonar.projectKey=FE_SLA_${bamboo.aed.PLAN_NAME}
-Dsonar.sources=sla_top.sdk/R5_HiL_Replay_App/src
-Dsonar.cfamily.build-wrapper-output=build_output
-Dsonar.log.level=DEBUG
-Dsonar.cfamily.gcov.reportsPath=sla_top.sdk/R5_HiL_Replay_App

Whats can be the problem?

Greets

Hi @AED,

It means that the C/C++ analyzer was not able to analyze any file of your project, none of the entries in the build-wrapper-dump.json matched an entry on SonarQube. It means that you never had a successful analysis for your project, we added such protection in the most recent version in order to help user to get correct analysis.

Is everything running inside docker at the same os level? (build-wrapper + build + analysis) This is a requirement.

Hi Massimo,

yes, we mapping the build-wrapper into docker. With Version 7.9.1 its work but now we cant go back to 7.9.1. The other Projects running without problems with this docker and analyze it. Can we disable the c/C++ Analyze for testing?

The same output was in the work sample, but then sonar load after them another Plugins Java, gcov etc.

Hi @AED,

does it mean that build-wrapper runs inside docker?

If yes, are you running sonar-scanner inside docker?

Hi Massimo,
no the files are on the Host, but we mount from there /opt to every docker.

/opt/build-wrapper-linux-x86
/opt/sonar-scanner-4.2.0.1873-linux

We have ~ 60 Projects (~40 with this Docker) Sonarqube works, but only in the projects there are use xsdk its nothing work after the upgrade.

Hi there,

I’m a developer at AED, and will try to describe the issue more precisely.

We are using sonarqube to analyze source code in multiple languages, out of these also C and C++.
For most build plans, everything is working fine, we only encounter problems when using the build-wrapper together with with the “xsdk” Framework (Software Development Kit (SDK)).

If we track a normal Makefile build with the build-wrapper, the code is analyzed correctly, as in the following command line:

/opt/build-wrapper-linux-x86/build-wrapper-linux-x86-64 --out-dir ${bamboo.build.working.directory}/build_output make

But if we compile C code using xsdk, the build-wrapper seems to not track the build correctly. We use the following line to trigger the build:

/opt/build-wrapper-linux-x86/build-wrapper-linux-x86-64 --out-dir build_output xsdk -batch -source ./build_r5_firmware.tcl

Result: The folder build_output is created, but the build-wrapper-dump.json is only ~40KB, and seems to not contain information about the compiled files. The build then fails with the message posted by Steffen above:

error 05-Nov-2019 11:37:30 java.lang.IllegalStateException: The “build-wrapper-dump.json” file was found but 0 C/C++/Objective-C files were analyzed

As xsdk is based on Eclipse, and uses a gcc compiler during the build, we would have expected that the build-wrapper can track our build. What can we do to make the build-wrapper track the build correctly?

Best,

Tom

Hi @thomas.rehner,

thank you for the very good explanation.

These could be a couple of hints to check:

  • do you know if xsdk works in daemon mode? build-wrapper follows children processes, build processes should not be ran in daemon mode otherwise build-wrapper would not be able to observe created processes
  • the gcc compiler used by xsdk is statically linked (unlikely to happen)

Hi Massimo,

thanks for the quick response.

We used top to check which processes are started by xsdk. Here is a snapshot during the build:

top - 09:30:55 up 60 days, 20:43,  0 users,  load average: 0.00, 0.03, 0.07
Tasks:  16 total,   2 running,  13 sleeping,   0 stopped,   1 zombie
%Cpu(s):  5.1 us,  1.5 sy,  0.0 ni, 93.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13141064+total, 41253128 free,  5657160 used, 84500360 buff/cache
KiB Swap: 15624188 total, 15602536 free,    21652 used. 12292318+avail Mem 

   PID USER      PR  NI S  %CPU %MEM     TIME+ COMMAND                                                                                                                                                                                                                                                           PPID   PGRP
  6579 root      20   0 S   4.0  0.2   0:07.61 java                                                                                                                                                                                                                                                              6578   6452
  6780 root      20   0 R   3.0  0.0   0:00.03 cc1                                                                                                                                                                                                                                                               6779   6636
     1 root      20   0 S   0.0  0.0   0:00.17 bash                                                                                                                                                                                                                                                                 0      1
   113 root      20   0 S   0.0  0.0   0:00.00 ssh-agent                                                                                                                                                                                                                                                            1    113
   119 root      20   0 S   0.0  0.0   0:00.06 gpg-agent                                                                                                                                                                                                                                                            1    119
  6452 root      20   0 S   0.0  0.0   0:00.00 xvfb-run                                                                                                                                                                                                                                                             1   6452
  6463 root      20   0 S   0.0  0.0   0:00.06 Xvfb                                                                                                                                                                                                                                                              6452   6452
  6466 root      20   0 R   0.0  0.0   0:00.01 top                                                                                                                                                                                                                                                                  1   6466
  6468 root      20   0 S   0.0  0.0   0:00.00 xsdk                                                                                                                                                                                                                                                              6452   6452
  6489 root      20   0 S   0.0  0.0   0:00.01 loader                                                                                                                                                                                                                                                            6468   6452
  6519 root      20   0 S   0.0  0.0   0:00.56 rdi_xsct                                                                                                                                                                                                                                                          6489   6452
  6525 root      20   0 Z   0.0  0.0   0:00.00 xsdk                                                                                                                                                                                                                                                              6519   6452
  6578 root      20   0 S   0.0  0.0   0:00.01 eclipse                                                                                                                                                                                                                                                              1   6452
  6636 root      20   0 S   0.0  0.0   0:00.59 make                                                                                                                                                                                                                                                              6579   6636
  6777 root      20   0 S   0.0  0.0   0:00.00 make                                                                                                                                                                                                                                                              6636   6636
  6779 root      20   0 S   0.0  0.0   0:00.00 armr5-none-eabi                                                                                                                                                                                                                                                   6777   6636

The last column shows: The make and gcc (armr5-none-eabi) processes are running in process group id 6636,while xsdk and various other processes are in 6452. From your comment, I’d assume this is the root cause why the build-wrapper doesn’t track the build.

Can we do anything about this?

Best,

Tom

Hi @thomas.rehner,

if you confirm that it is running in daemon mode, could you check if there is a way to run it in non-daemon mode?

Hi,
we don’t start xsdk in daemon mode. I couldn’t find out yet how to achieve that the make and gcc are not running in a different process group, and if it is possible to change this - I guess it happens somewhere in the eclipse process?
Is there any way to make the build-wrapper aware of the process if we can’t change the way “xsdk” works?
Best,
Tom

Hi @thomas.rehner,

yes, you should check if there is any option to not spawn a daemon like process.

No, the technology used allows to follow created processes but not to attach to processes on another process tree.