Sonar fails to parse standard headers from latest VC++ release (17.10.4)

SonarQube Cloud
,
1

Hello,

Our code coverage measurement a suddenly dropped to 0% for all our C++ projects.

Something happened on last Monday 26th between 17h20 (GMT+2) and 17h53. The day after, I received an email explaining me our subscription had been updated to SonarQube Cloud Team on the 26th.

The reason why I think it is related is because all our 4 GitLab runners (on Windows) have been impacted at the same time without touching anything on our CI regarding this aspect.

The issue we face is with the C++ code coverage evaluation from bullseye XML output. More specifically, the regex given to sonar.test.inclusions seems to behave differently now and to ignore all our source code rather than some specific files.

Here is our config:

sonar.sources=code
sonar.exclusions=code/tools/**/*

sonar.tests=code
sonar.test.inclusions=code/**/test/**/*,code/tests/**/*

Before the update, we had the following output in our logs:

INFO: ------------- Run sensors on project
INFO: Sensor bullseye [cpp]
INFO: Parsing C:\buildroot\projectname\sonar\coverage_report.xml
WARN: File not analysed by Sonar, so ignoring coverage: c:\buildroot\projectname\code\..\build\Debug\CMakeFiles\3.29.6\CompilerIdC\CMakeCCompilerId.c
WARN: File not analysed by Sonar, so ignoring coverage: c:\buildroot\projectname\code\..\build\Debug\CMakeFiles\3.29.6\CompilerIdCXX\CMakeCXXCompilerId.cpp
WARN: File not analysed by Sonar, so ignoring coverage: c:\buildroot\projectname\code\..\build\Debug\CMakeFiles\ShowIncludes\main.c
[...]
WARN: File not analysed by Sonar, so ignoring coverage: c:\buildroot\projectname\code\..\ext\submodule_project\code\libname\src\SomeFile.cpp
[...]

Now we have the same output, but with more lines (our source code that is now ignored in the coverage):

WARN: File not analysed by Sonar, so ignoring coverage: c:\buildroot\projectname\code\libname\src\units\UnitName\AccuracyCheckUnitFile.cpp
[...]

Any idea how to fix this?

Thanks.

2

I played a bit with the config that I have simplified to this one:

sonar.sources=code
sonar.cfamily.compile-commands=sonar/compile_commands.json
sonar.testExecutionReportPaths=sonar/merged_sonar_test_reports.xml
sonar.cfamily.bullseye.reportPath=sonar/coverage_report.xml

and here’s the output I get for ErrorManagement.cpp as an example (the coverage is still ignored):

[10/128] Building CXX object code\base\CMakeFiles\lupin_core__base.dir\src\ErrorManagement.cpp.obj
BullseyeCoverage Compile 9.15.6 Windows License XXXXX, Copyright (c) Bullseye Testing Technology
[...]
INFO: Parsing C:\buildroot\myproject\sonar\merged_sonar_test_reports.xml
WARN: Line 49 of report refers to a file which is not configured as a test file: C:\buildroot\myproject\code\base\test\ErrorManagement_test.cpp
[...]
INFO: Number of remaining compilation units to be analyzed: 102
INFO: Detected standards: c++20=102
INFO: Starting the analysis of 102 CFamily compilation units
INFO: [6/102] Cache hit for: C:\buildroot\myproject\code\base\src\ErrorManagement.cpp
[...]
INFO: ------------- Run sensors on project
INFO: Sensor bullseye [cpp]
INFO: Parsing C:\buildroot\myproject\sonar\coverage_report.xml
WARN: File not analysed by Sonar, so ignoring coverage: c:\buildroot\myproject\code\base\src\ErrorManagement.cpp

The source file is analyzed but then ignored in the coverage output.

3

Hello,

I’ve tried to clear the cache on the runner and to update sonar-scanner (from 5.0.1.3006) to 7.1.0.4889, but the issue is still here.

Any idea what I could do?

5

Hey @aurelienlupin

Sorry this got lost last week.

This should purely be a coincidence. Nothing about the plan you’re using should affect coverage!

I’ve flagged this for attention.

The one thing that jumped out at me is that the case of your drive name appears to be different c:\ in your coverage report as opposed to C:\ everywhere else.

Not sure that should matter on Windows, but it would be interesting to know if that changed around the time you noticed the issue.

6

Hello Colin,

Thanks for your answer. Yes there’s a difference in the case of the drive name, but it was already there in past builds and did not pose a problem.

I took time today to check the whole diff produced on the same machine before and after without noticing anything new in the output.

I said we didn’t touch the runner regarding Sonar, but what was done that day was to run Windows Update and update the compiler version (install latest VC++). I though it would not impact sonar-scanner, but now I think the issue is with the latest version of VC++.

Indeed, I ran sonar-scanner with the --debug flag this time, and guess what I have: many DEBUG warnings about a failure to parse code in the standard headers:

17:50:44.401 DEBUG C:\buildroot\src\main.cpp:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include\type_traits:1890 - '_ClassTy' does not refer to a value
17:50:44.401 DEBUG C:\buildroot\src\main.cpp:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include\type_traits:1895 - '_ClassTy1' does not refer to a value
17:50:44.402 DEBUG C:\buildroot\src\main.cpp:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include\type_traits:1895 - '_ClassTy2' does not refer to a value
17:50:44.402 DEBUG C:\buildroot\src\main.cpp:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include\atomic:256 - no type named '__builtin_zero_non_value_bits' in the global namespace
[...]
17:50:44.403 DEBUG C:\buildroot\src\main.cpp:C:\.conan\data\ms-gsl\4.0.0\soup\rc1\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\gsl\util:77 - expected string literal as argument of 'suppress' attribute
[...]
17:50:44.403 DEBUG C:\buildroot\src\main.cpp:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include\__msvc_string_view.hpp:449 - use of undeclared identifier '__builtin_u8strlen'
17:50:44.403 DEBUG C:\buildroot\src\main.cpp:C:\.conan\data\ms-gsl\4.0.0\soup\rc1\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\gsl\span:303 - expected string literal as argument of 'suppress' attribute
[...]
17:50:44.403 DEBUG C:\buildroot\src\main.cpp:C:\.conan\data\ms-gsl\4.0.0\soup\rc1\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\gsl\algorithm:52 - expected string literal as argument of 'suppress' attribute
17:50:44.403 DEBUG C:\buildroot\src\main.cpp:C:\.conan\data\ms-gsl\4.0.0\soup\rc1\package\5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9\include\gsl\string_span:112 - expected string literal as argument of 'suppress' attribute
[...]

So here’s my guess: the Sonar C++ parser can no longer parse some standard headers that come with the latest version of VC++ such as <type_traits> or <atomic> (and also seems to have issues with the GSL which has not been updated).

Since we include these headers everywhere via our precompiled header file, this prevents Sonar to parse all our code. Which is why we don’t have the coverage later on because File not analysed by Sonar.

For some reason, this parsing error is reported as a DEBUG issue which makes it invisible by default (if -X is not specified).

I’ll try to compile a simpler example to confirm that it works on code that do not include the problematic headers.

PS: I edited the title of this topic to reflect this discovery

7

And here I was thinking I was onto something :laughing:

Thanks for the further investigation. I will let our devs take it from here.

1 Like
10

Hello @aurelienlupin,

I have been investigating your issue a bit and unfortunately I have not been able to reproduce it.

Doing some preliminary narrowing down:

  • It doesn’t seem to originate in the sonar-scanner because you mentioned that the issue first appeared with an older version, which you hadn’t upgraded on D-day, and persisted even after upgrading.
  • It could originate in the CFamily and/or Bullseye sensor, which are responsible for parsing the coverage results and integrating them into the analyzed files. Indeed, they were updated around D-day, but I checked and the Bullseye driver has not undergone any changes in the past few versions.

Should it be either of the above two parts, it would be a regression affecting more users, and we have not received any further reports.

So, we are left with the last moving part, which is probably some difference in your CI environment that you are not aware of, or that you do not directly link to the issue. Note that I say probably because I could be wrong, but let’s start by first analyzing the most probable cause.

Concerning your first assumption that it could be associated with an update of the compiler and standard headers: since it is something that changed on D-day, we can’t completely rule it out. But I’m afraid it’s not related to what you mention about the parser not being able to parse the headers and therefore having no coverage information. In that case, all the issues would have suddenly disappeared from SonarQube as well, and not just the coverage. Can you confirm that you are still seeing the issues and that there was no noticeable difference on D-day compared to the previous day (in terms of issues)?

This warning seen in debug mode File not scanned by Sonar, so coverage is ignored is not saying that the file was attempted to be analyzed but failed, but that the file is not in the set of files to be analyzed. This set is calculated by the scanner from sonar.sources, sonar.inclusions and sonar.exclusions. This warning implies that the file referenced in the coverage report is not indexed by the scanner, so it has not been analyzed.

But if you verify that it is indeed analyzed, by looking for an entry of the form Cache hit for and/or checking if there is any issue raised in that file, it leads me to think that there is a mismatch between the file path as seen from the scanner, and the one specified in the coverage report and that is used by the bullseye sensor.

Can you check if the file path indexed by the sensor (Cache hit for: <path>) is exactly the same as the one in the coverage report (File not analysed by Sonar, so ignoring coverage: <path>)? Just for context, do any of these paths contain substituted drives or symbolic links?

Let’s make sure of that before proceeding with the investigation.

11

Hello Guillem,

Thanks for your answer.

I have simplified as much as possible what we do in an example that I have pushed here: Aurelien Regat-Barrel / demo-sonar-cpp-coverage-issue · GitLab

As you can see, it is a very simple C++ app built with cmake on Windows. And a simple sonar-project.properties config.

In the folder ci_output you will get all the output: from the CI pipeline, from SonarCloud dashboard and from BullseyeCoverage (xml used by sonar-scanner + html output).

In this example, the source code is properly parsed with success and Sonar successfully reports some issues. But the coverage is not reported. So your analysis seems to be correct.

BullseyeCoverage is able to identify and report a coverage of 50% of the code. But sonar-scanner is still not happy in the end:

16:11:46.191 DEBUG Cache miss for C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp: file not in cache
16:11:46.194 INFO  [thread-1] Started analyzing: C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp
16:11:46.337 INFO  [1/1] Finished analyzing: C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp (123ms)
16:11:46.338 DEBUG Java memory after analysis is done (used, peak): 123, 123
16:11:46.343 DEBUG C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp not marked as unchanged: file content changed

[...]
16:11:46.473 WARN  File not analysed by Sonar, so ignoring coverage: c:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp

here are the 2 paths (analyzed and checked for coverage):

C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp
c:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\code\main.cpp

So only the drive letter case is different:

  • this was already the case in the past and did pose an issue so far
  • if the issue is here, how can I change this part as I am specifying file patterns relative to the git repo

Thanks for your help.

12

Hello Aurelien,

Thank you for crafting the minimized example and for the care you have taken to include so much important information!

Unfortunately, even with all this extra information I have not been able to reproduce the exact same problem you experience.

Note: The sonar-project.properties file is not complete, I assume you set the project organization and key using environment variables.

You can check out one of my closest attempts here:

Note that it should be almost the same as you have. I have manually edited the path in coverage_report.xml.

I suspect it must have something to do with some slight differences in the way the sonar-scanner and cfamily sensor see file paths in the presence of filesystem decorators (symbolic links and substituted drives). But I can’t know for sure because I don’t have visibility over the exact configuration of your execution environment, which I believe may be a self-managed GitLab runner or a GitLab runner with a custom image.

For example, I was able to get a similar failure by invoking the sonar-scanner from a directory junction C:\lupin_core pointing to C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core. But that shouldn’t be your case because the line Project root configuration file: C:\GitLab-Runner\builds\vVSLbBoe\0\lupindental\arsene\software\common\lupin_core\sonar-project.properties tells us that it was invoked from the real directory.

  • Do you know if any of the paths involved contain filesystem decorators? This part vVSLbBoe\0\lupindental\arsene\software\common makes me think there may be some.
  • Have you tried to reproduce this minimized example locally (not in your CI environment)?

Sorry to get back to you with no remediation yet, but understand that it is difficult to debug it without access to the actual environment.

We can continue the discussion in private if you deem it appropriate.

14

Hello,

Our CI servers are dedicated Windows Server machines from OVH, with 2 disks in software mirror. There’s nothing fancy for the C:\ volume, the build directory does not contain any link or junction point, the weird name is generated by GitLab runner as a root dir.

BTW, the code coverage is back to normal! We did nothing special on our side, it just came back.

I think it was related to the update of VC++ because we had another server where VC++ was not updated, and it has not been impacted by this issue. But now everything works well again so I’ll close this issue.

Thanks for the effort to analyze it :slight_smile:

1 Like