SonarScanner fails after updating Visual C++ to 17.9.1

aurelienlupin · February 23, 2024, 6:02pm

Hello,

we run sonar-scanner 5.0.1.3006 from our GitLab runner on Windows to analyze C++ builds (via cmake compile commands). So far it worked well, since many months.

We regularly update Visual Studio on the runners, without any issue. But I just updated it now (to version 17.9.1) and sonar-scanner is now failing with the following exception:

$ sonar-scanner
INFO: Scanner configuration file: C:\GitLab-Runner\sonar\sonar-scanner-5.0.1.3006-windows\bin\..\conf\sonar-scanner.properties
INFO: Project root configuration file: XXXXX\sonar-project.properties
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.7 Eclipse Adoptium (64-bit)
INFO: Windows Server 2022 10.0 amd64
INFO: User cache: C:\.sonar\cache
INFO: Analyzing on SonarCloud
INFO: Default locale: "en_US", source code encoding: "windows-1252" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=151ms
INFO: Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
INFO: User cache: C:\.sonar\cache
INFO: Loading required plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=78ms
INFO: Load/download plugins
INFO: Load/download plugins (done) | time=147ms
INFO: Found an active CI vendor: 'Gitlab CI'
INFO: Load project settings for component key: 'XXX'
INFO: Load project settings for component key: 'XXX' (done) | time=60ms
INFO: Process project properties
INFO: Project key: XXX
INFO: Base dir: XXXXX
INFO: Working dir: XXXX\.scannerwork
INFO: Load project branches
INFO: Load project branches (done) | time=74ms
INFO: Check ALM binding of project 'XXX'
INFO: Detected project binding: BOUND
INFO: Check ALM binding of project 'XXX' (done) | time=51ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=62ms
INFO: Load branch configuration
INFO: Auto-configuring branch 150-add-metrics-support-in-telemetry-backend-with-simple-text-export
INFO: Load branch configuration (done) | time=2ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=129ms
INFO: Load active rules
INFO: Load active rules (done) | time=2339ms
INFO: Organization key: XXX
INFO: Branch name: 150-add-metrics-support-in-telemetry-backend-with-simple-text-export, type: short-lived
INFO: Preprocessing files...
INFO: 1 language detected in 135 preprocessed files
INFO: 141 files ignored because of inclusion/exclusion patterns
INFO: 0 files ignored because of scm ignore settings
INFO: Loading plugins for detected languages
INFO: Load/download plugins
INFO: Load/download plugins (done) | time=6ms
INFO: Load project repositories
INFO: Load project repositories (done) | time=159ms
INFO: SCM collecting changed files in the branch
INFO: SCM collecting changed files in the branch (done) | time=90ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Excluded sources: **/build-wrapper-dump.json, code/tools/**/*, code/**/test/**/*, code/tests/**/*
INFO:   Included tests: code/**/test/**/*, code/tests/**/*
INFO: 135 files indexed
INFO: Quality profile for cpp: LupinCpp
INFO: ------------- Run sensors on module lupin_core
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=59ms
INFO: Sensor cache enabled
INFO: Load sensor cache
INFO: Load sensor cache (54 KB) | time=381ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=2ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: 125 source files to be analyzed
INFO: 125/125 source files have been analyzed
INFO: Sensor TextAndSecretsSensor [text] (done) | time=782ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=53ms
INFO: Sensor Generic Test Executions Report
INFO: Parsing XXX\sonar\merged_sonar_test_reports.xml
INFO: Imported test execution data for 25 files
INFO: Sensor Generic Test Executions Report (done) | time=42ms
INFO: Sensor CFamily [cpp]
INFO: CFamily plugin version: 6.53.0.67215
INFO: Using compile commands: XXX\sonar\compile_commands.json
INFO: Available processors: 16
INFO: Using 16 threads for analysis.
INFO: Loading cache from: server
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 8.489s
INFO: Final Memory: 15M/80M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
java.lang.IllegalStateException: unexpanded response file
	at com.sonar.cpp.analyzer.MsvcDriver.getOptionsFromCmd(MsvcDriver.java:483)
	at com.sonar.cpp.analyzer.MsvcDriver.onCapture(MsvcDriver.java:133)
	at com.sonar.cpp.analyzer.CompilerDrivers.onCapture(CompilerDrivers.java:36)
	at com.sonar.cpp.plugin.CFamilySensor.lambda$process$37(CFamilySensor.java:1447)
	at com.sonar.cpp.plugin.CompileCommandsReader.readCaptures(CompileCommandsReader.java:61)
	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:1445)
	at com.sonar.cpp.plugin.CFamilySensor.processConfigurationReader(CFamilySensor.java:750)
	at com.sonar.cpp.plugin.CFamilySensor.lambda$process$4(CFamilySensor.java:468)
	at java.base/java.lang.Iterable.forEach(Unknown Source)
	at com.sonar.cpp.plugin.CFamilySensor.process(CFamilySensor.java:468)
	at com.sonar.cpp.plugin.CFamilySensor.execute(CFamilySensor.java:241)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:62)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:51)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:64)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:192)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:188)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:159)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.bootstrap.ScannerContainer.doAfterStart(ScannerContainer.java:397)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:125)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:57)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:51)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at jdk.proxy1/jdk.proxy1.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:126)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:81)
	at org.sonarsource.scanner.cli.Main.main(Main.java:62)
ERROR: 
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

any idea about what’s going on / how to fix that?

michael.jabbour · February 25, 2024, 8:01pm

Hi @aurelienlupin, and welcome to the community.

If I understand correctly, you are using CMake to generate your compile_commands.json file. Unfortunately, we don’t support unexpanded response files when reading compilation commands at the moment, see [CPP-3572] - Jira.

You can instruct CMake to avoid generating response files by setting the following variables in your CMakeLists.txt file, and then regenerate the compilation database.

set(CMAKE_CXX_USE_RESPONSE_FILE_FOR_INCLUDES 0)
set(CMAKE_C_USE_RESPONSE_FILE_FOR_INCLUDES 0)

Let us know if this doesn’t fix your problem.

Best regards,
Michael

aurelienlupin · February 26, 2024, 8:47pm

Hello Michael,

Thanks for the answer. We indeed rely on compile commands, via Ninja generator. So far we did not have any issue related to response files, and adding the cmake variables you mentioned did not solve the issue.

However it helped me understand what happens. We rely on the cmake version provided by VC++, and it recently has been update to 3.28. And this versions adds support for C++ 20 modules. As a result, a “modmap” file is automatically generated by cmake Ninja generator (if building in C++ 20 mode as we do) and is automatically added to the command line as a response file.

I quickly tried to disable it (by testing CMAKE_CXX_SCAN_FOR_MODULES=OFF) but with no success. So what I did as a quick fix is to install cmake 3.27 on our runners and making sure to use this version. And it solves our issue

So, to recap the situation: since cmake 3.28, if compiling in C++ 20 mode, sonar-scanner can no longer process the compile commands file generated by cmake (at least with Ninja generator).

I think it’s an issue for your tool and would deserve a quick fix, such as ignoring “modmap” files passed on the command line as response files.

Regards,

michael.jabbour · February 27, 2024, 1:55am

Hello @aurelienlupin and thank you very much for the detailed investigation, we appreciate it.

I could reproduce the problem in a minimal project when using CMake 3.28.1, and it seems to have been fixed on 3.28.2 according to this issue. Could you confirm that you are using CMake version 3.28.1?

Best regards,
Michael

aurelienlupin · February 27, 2024, 8:46am

Yes indeed: cmake version 3.28.0-msvc1

system · March 6, 2024, 10:21am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.