I’m using SonarQube to analyze code written primarily in ROS’ C++ interface. A part of ROS’ build system involves converting .msg files into .h files and placing them in a dedicated installation directory. So after building, a file that was in the following location during compilation:
/src/package1/custom_msgs/msg/CustomMessage.msg
Will create an auto-generated .h file in the following location, able to be referenced through Catkin makefile magic:
My current issue is that SonarQube doesn’t know about the .h file during its scan and ends up with hundreds of warnings in the log to the tune of:
'custom_msgs/CustomMessage.h' file not found
And as a result of those missing files, the SonarQube analysis is severely crippled.
My question is, is there a way to give SonarQube information about the header file’s contents or new location? Preferably automatically since these messages are added and moved pretty frequently.
I have to ask the dumb question: Are you using the build-wrapper? I ask because it seems like those header files would be found during compilation & thus the build-wrapper, and by extension analysis, would know where they are…?
The supplied property files is as follows, with sensitive information removed:
# must be unique in a given SonarQube instance
sonar.projectKey=XXXXX
sonar.sources=src
sonar.host.url=XXXXX
sonar.login=XXXXX
sonar.cfamily.compile-commands=build/compile_commands.json
sonar.exclusions=**/*.doc,**/*.xml
sonar.cfamily.build-wrapper-output=bw-output
sonar.cfamily.threads=6
#Disable caching
sonar.cfamily.cache.enabled=false
# Encoding of the source code. Default is default system encoding
sonar.sourceEncoding=UTF-8
And then the resultant SonarQube analysis is on the web GUI where I’d expect it to be, where it’s right enough to prove it has most of the insight I’d expect it to have, but wrong often enough that I think it’s missing critical insight. These missing .h files seem like a likely culprit. See my linked issue for more details on the actual issue I’m trying to solve.
If I’m following your question, Colcon isn’t a compiler. It’s a command line tool to improve the building workflow. We use it to cleanly build ROS code.
colcon is a command line tool to improve the workflow of building, testing and using multiple software packages. It automates the process, handles the ordering and sets up the environment to use the packages.
I’m not sure I understand what exactly I need to wrap. The colcon command isn’t a compiler, but it runs the thousands of auto-generated compilation commands like the following (pulled from the archived compile_commands.json)
So all of the compilation is happening in the colcon command that I have wrapped by the build wrapper, and the individual commands should be supported build commands. It’s similar to cmake/make/meson: not technically a compiler, but very close to the operation.
In your config file, I can see you specify both the compilation database (sonar.cfamily.compile-commands) and the output of the build wrapper (sonar.cfamily.build-wrapper-output), but only one is needed.
Can you try with only the compilation database? If you still get the same error, sharing compile_commands.json and the debug logs of the scanner would be useful. If you don’t want them to be public, I can send you a private message.
I attempted running with the same command and properties file with the build-wrapper-output line commented out and that resulted in this error message explicitly asking me to use the build wrapper:
15:44:43.925 INFO: ------------------------------------------------------------------------
build 08-Feb-2023 09:44:43 15:44:43.925 INFO: EXECUTION FAILURE
build 08-Feb-2023 09:44:43 15:44:43.925 INFO: ------------------------------------------------------------------------
build 08-Feb-2023 09:44:43 15:44:43.926 INFO: Total time: 21.127s
error 08-Feb-2023 09:44:44 15:44:44.120 ERROR: Error during SonarScanner execution
build 08-Feb-2023 09:44:44 15:44:44.120 INFO: Final Memory: 144M/487M
build 08-Feb-2023 09:44:44 15:44:44.120 INFO: ------------------------------------------------------------------------
error 08-Feb-2023 09:44:44 java.lang.UnsupportedOperationException:
error 08-Feb-2023 09:44:44
error 08-Feb-2023 09:44:44 The only way to get an accurate analysis of C/C++/Objective-C files is by using the SonarSource build-wrapper
error 08-Feb-2023 09:44:44 and setting the property "sonar.cfamily.build-wrapper-output", but it was not specified.
error 08-Feb-2023 09:44:44
error 08-Feb-2023 09:44:44 If you don't want to analyze C/C++/Objective-C files, then prevent them from being analyzed by setting the following properties:
error 08-Feb-2023 09:44:44
error 08-Feb-2023 09:44:44 sonar.c.file.suffixes=-
error 08-Feb-2023 09:44:44 sonar.cpp.file.suffixes=-
error 08-Feb-2023 09:44:44 sonar.objc.file.suffixes=-
error 08-Feb-2023 09:44:44
error 08-Feb-2023 09:44:44
error 08-Feb-2023 09:44:44 at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:210)
error 08-Feb-2023 09:44:44 at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:177)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:62)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)
error 08-Feb-2023 09:44:44 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
error 08-Feb-2023 09:44:44 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:392)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:388)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:357)
error 08-Feb-2023 09:44:44 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
error 08-Feb-2023 09:44:44 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
error 08-Feb-2023 09:44:44 at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:150)
error 08-Feb-2023 09:44:44 at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)
error 08-Feb-2023 09:44:44 at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)
error 08-Feb-2023 09:44:44 at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
error 08-Feb-2023 09:44:44 at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
error 08-Feb-2023 09:44:44 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
error 08-Feb-2023 09:44:44 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
error 08-Feb-2023 09:44:44 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
error 08-Feb-2023 09:44:44 at java.base/java.lang.reflect.Method.invoke(Unknown Source)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
error 08-Feb-2023 09:44:44 at com.sun.proxy.$Proxy0.execute(Unknown Source)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
error 08-Feb-2023 09:44:44 at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Before I check with my people to see if I can even send the commands file to you, can you confirm that I should be able to even do this? The error message seems pretty explicit when it says that I cannot.
Sorry, you were using SonarQube LTS 8.9.10, right? The bundled C++ analyzer (6.20) does not support the compilation database.
To better understand why the analyzer is missing those headers we need to have a look then at the files build-wrapper-dump.json and build-wrapper.log stored inside bw-output.
Sorry for the delay, I wasn’t able to secure permission to share our wrapper logs due to the nature of the project being worked on. However, I did experiment and determined that moving the code to an install directory in the build command appears to cause the issue. Details are here: SonarQube support for --install flag in catkin/colcon