Scanner Scan code failed

SonarQube Server / Community Build
,
1

Last week, we purchased sonarqube developer. I had a problem scanning the C code. I followed the steps of the official website strictly, but there were problems that I could not handle. I’ll give you the screenshot of my error and relevant information. Can you help me solve this problem?
sonarQube Version: sonarqube-9.6.0.59041

scanner: sonar-scanner-4.6.2.2472-linux

CFamily plugin version: 6.36.0.52033

Sonar start command: sonar-start.txt

build-wrapper & bw-output & sonar-scanner: sonar.zip

image
image1716×862 303 KB

image
image1716×862 325 KB

Sonar start command: sonar-start.txt

echo "$(sed 's/8.8.8.8/192.168.0.112/g' /etc/resolv.conf)" > /etc/resolv.conf
cd /home/zhaobingfeng/lds-wifi-platform-v2; wget http://192.168.5.33:59980/lds-wifi-platform-v2-sonar.sh
chown zhaobingfeng:zhaobingfeng lds-wifi-platform-v2-sonar.sh
chmod +x lds-wifi-platform-v2-sonar.sh
su zhaobingfeng -s lds-wifi-platform-v2-sonar.sh




#start.sh
#!/bin/bash
echo "++++++sonar扫描++++++开始编译++++++"
mkdir -p .sonar
curl -sSLo build-wrapper-linux-x86.zip  http://192.168.5.207:9000/static/cpp/build-wrapper-linux-x86.zip
unzip -o build-wrapper-linux-x86.zip -d .sonar
repo sync
make clean
.sonar/build-wrapper-linux-x86/build-wrapper-linux-x86-64 --out-dir .sonar/bw-output make <<EOF
6
n
n
n
EOF


echo "++++++sonar扫描++++++扫描代码++++++"
curl -sSLo sonar-scanner.zip http://192.168.5.33:59980/sonar-scanner.zip
unzip -o sonar-scanner.zip -d .sonar
a=`ls output/build/ | grep 'lds-*' -G | tr '\n' ',' | sed 's/lds-/output\/build\/lds-/g'`
.sonar/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner -Dsonar.cfamily.build-wrapper-output=.sonar/bw-output -Dsonar.projectKey=lds-wifi-platform-v2 -Dsonar.sources=$aapps,components -Dsonar.host.url=http://192.168.5.207:9000 -Dsonar.login=sqp_0f2cea007637c690a2c7d2396ca0e40a3c89d64b -Dsonar.exclusions=**/*.java,**/*.py,**/*.xmind,**/*.rb -X
2

Hey there.

I’m not clever enough to be able to decipher this line – what is it doing, exactly?

3

This sentence is to eliminate unnecessary scanning paths. You can see that the next sentence will be spliced. Note:$a

.sonar/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner -Dsonar.cfamily.build-wrapper-output=.sonar/bw-output -Dsonar.projectKey=lds-wifi-platform-v2 -Dsonar.sources=$aapps,components -Dsonar.host.url=http://192.168.5.207:9000 -Dsonar.login=sqp_0f2cea007637c690a2c7d2396ca0e40a3c89d64b -Dsonar.exclusions=**/*.java,**/*.py,**/*.xmind,**/*.rb -X

I originally wanted to use build-wrapper.log and build-wrapper-dump JSON files are sent out, but they are too large. The website does not support such large data. If you want to check, which part do I need to cut?

4

hi Colin,

In another project, I encountered the same problem.

I think like this: the C & C + + language has a huge SDK, and the code we develop is only a small part, so we must specify that only a few code paths should be scanned, otherwise we will fall into endless bug false positives. But at this time, the build wrapper can’t parse out the relevant path, resulting in the inability to scan the file when the sonar scanner scans. At this time, the problem arises in the build wrapper?

Of course, it may also be my own configuration problem. Have you encountered similar problems?

This time, build-wrapper-dump.json and build-wrapper.log is also very large, and it is impossible to paste the file into the web page. If it is convenient, is there any other way to contact and communicate?

start.sh

#1.build_all,生成build-wrapper文件
curl -sSLo build-wrapper-linux-x86.zip  http://192.168.5.207:9000/static/cpp/build-wrapper-linux-x86.zip
unzip -o build-wrapper-linux-x86.zip -d .sonar
chmod +x ./ -R
cd lds_ipc/
../.sonar/build-wrapper-linux-x86/build-wrapper-linux-x86-64 --out-dir ../.sonar/bw-output ./build_scripts/m219_build.sh

#2.sonar扫描
cd ../
curl -sSLo sonar-scanner.zip http://192.168.5.33:59980/sonar-scanner.zip
unzip -o sonar-scanner.zip -d .sonar
.sonar/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner \
    -Dsonar.cfamily.build-wrapper-output=.sonar/bw-output \
    -Dsonar.projectKey=$SONAR_PROJECT_KEY \
	-Dsonar.sources=platform/source/app/lds_application,platform/source/app/lds_system_server,platform/source/app/lds_ipc,platform/source/lib/lds_libwifiapi,platform/source/lib/lds_ringbuffer,platform/source/lib/lds_storage \
    -Dsonar.host.url=$SONAR_HOST_URL \
    -Dsonar.login=$SONAR_TOKEN \
    -Dsonar.qualitygate.wait=true \

We look forward to your reply

5

The third project has the same problem. This time, his log and JSON file are relatively small. I can upload them. You can refer to this reply directly.
#.gitlab-ci.yml

sonar_scan_analyze:
  script:
    - sh ci/sonar_scan_analyze.sh
  only:
    - schedules
    - master

#ci/sonar_scan_analyze.sh

#!/usr/bin/tmux
#1.build_all,生成build-wrapper文件
#修改launch.sh
cp ./launch.sh launch_sonar_start.sh
sed -i '6,49d' launch_sonar_start.sh
sed -i '5a\./m219e_build.sh' launch_sonar_start.sh
#修改base.sh和
cp /products/build/base.sh /products/build/true_base_for_sonar.sh
sed -i '5,22d' /products/build/base.sh.sh
#build-wrapper扫描
chmod +x ./ -R
curl -sSLo build-wrapper-linux-x86.zip  http://192.168.5.207:9000/static/cpp/build-wrapper-linux-x86.zip
unzip -o build-wrapper-linux-x86.zip -d .sonar
.sonar/build-wrapper-linux-x86/build-wrapper-linux-x86-64 --out-dir .sonar/bw-output ./launch_sonar_start.sh

#2.sonar扫描
curl -sSLo sonar-scanner.zip http://192.168.5.33:59980/sonar-scanner.zip
unzip -o sonar-scanner.zip -d .sonar
.sonar/sonar-scanner-4.6.2.2472-linux/bin/sonar-scanner \
    -Dsonar.cfamily.build-wrapper-output=.sonar/bw-output \
    -Dsonar.projectKey=$SONAR_PROJECT_KEY \
	-Dsonar.sources=. \
    -Dsonar.host.url=$SONAR_HOST_URL \
    -Dsonar.login=$SONAR_TOKEN \
    -Dsonar.qualitygate.wait=true \

build-wrapper-1.log (6.1 MB)
build-wrapper-2.log (6.1 MB)
build-wrapper-dump.json (5.3 MB)
Directory screenshot

image
image701×500 22.6 KB

Screenshot of gitlab CI failure
image
image1297×680 30 KB

6

Hey there.

Can you attach the full analysis logs (from GitLab CI) for the relatively small project? Even better if they are in debug mode (sonar-scanner -X).

7

hi
I don’t have any smaller projects on hand at present. This is the gitlab CI log of the third project and sonar-scanner -X
You don’t have to care about tput: No value for $TERM and no -T specified. This problem is caused by the lack of interactive shell in gitlab Ci, which does not affect the successful compilation of the project.
Thank you for your answer!
gitlab-ci.log (1.1 MB)

9

Hello @dr.missname,

We think we understand what is the problem. Usually, during the build, the source files that you checked out are directly compiled, and we use that information to know how to analyze them.

In your case, your build process seems to work a little bit differently. Instead of directly compiling your .c files, it does a 2-steps process:

  • It first generates a file that contains the output of the preprocessor applied to your source file (with a .c.c extension), by launching the compiler with the -E command line option.
  • It then compiles this preprocessed output

The build-wrapper-dump.json correctly tracks those two steps, but then the following problem happens:

  • When we find a command that contains the -E option, we skip it, because we are only interested in real compilation
  • When we find a command that contains the real compilation, we skip it, because it applies to a temporary file that is not part of your source code

So, all in all, we skip all files, and don’t analyze anything.

How can this be solved?

  • Maybe the best option is to revisit why you are compiling using this unusual 2-steps process. If you used a traditional build process, in addition to the fact that we would be able to analyze your code, there is a chance that you would get a faster build, because preprocessed files are notoriously large and cumbersome.

  • If the first option is not feasible to you, there might be a workaround: You can write a script that, after the build-wrapper-dump.json is generated, removes from it all the "-E", lines that it contains. The resulting json should look similar enough to the json that would be generated with a single-step compilation.

Hope this helps, let us know what you did!

11

Very nice! Thank you for your reply. Deleting the -E parameter successfully solved my problem. Thank you for your professional answer.

In addition, most of our company’s C / C + + code is based on IOT projects. there are many project groups and many compilation types are not uniform. I think, most of sonar’s implementations are application, the types are relatively uniform, right?

12

It’s hard to say because most of our users don’t tell us exactly what kind of code they analyze, but we have many questions coming from developers who use compilers for embedded software.

13

Thank you again for your professional reply, which helped me solve the problem that has plagued me for two weeks. So far, I have no new questions, and this problem can be closed.

14

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.