If I generate code coverage reports of multiple formats for the same code, is SonarQube able to combine the results of both report formats?
For example, say I have a program that runs one block of code (say 50% of the code) if running on Linux and another if running on Windows (the other 50%). Then I compile with gcc and generate gcov reports on Linux and compile with Visual Studio’s compilier and generate coveragexml reports on Windows after running my test suites. Then I move the reports to a central location, specifying in the sonar-project.properties the locations of the reports as specified here: https://docs.sonarqube.org/latest/analysis/coverage/
Will SonarQube be able to combine the results from both these report sources and show a test coverage of 100%?
SonarQube will merge the reports, line coverage is additive and condition coverage will retain the max. Condition coverage will retain the max, it is currently not possible to merge condition coverage because in the condition reports there is no information about which condition is covered.
it is currently not possible to merge condition coverage
Could you elaborate a bit on what you mean by “condition coverage” (if statements, #ifdef_WIN64 blocks, etc.) and how that differs from “line coverage”?
condition coverage is when the code can take different branches, like if statements, if you have if (a) {...} there are two possible paths, when a is true and when a is false. Coverage reports for this case only contain the number of covered branches, like 1 covered out of 2. If you have two reports both containing 1 covered out of 2 you actually don’t know if it is because of a being true or false, this is why the max is taken, simply 1 covered out of 2.
@mpaladin Thanks. However I’m still stuck on trying to merge reports of different formats. Reiterating the example I gave in the OP, I have a report generated with Visual Studio 2017 (.xml). I also have a a gcov reports generated with gcc (.gcov). Obviously, these were compiled and built separately and each using the build wrapper of their respective platforms, and creating their respective bw-outputs outputs directories. However in order to run the sonar-scanner, I need the reports of both formats to be on the same computer.
I tried and add my coverage.xml to my Linux build directory and then point to this report using sonar.cfamily.vscoveragexml.reportsPath, as well as my gcov reports using sonar.cfamily.gcov.reportsPath. However it just reported a coverage of zero on the SonarQube front end, which is not correct.
I understand this is a bit messy to explain so please let me know if I can clarify anything. At this point, it seems like SonarQube is not able to merge reports of different types like you mentioned in your first reply. If it really is possible, there are a lot of things that aren’t clear: for example, which bw-outputs folder should I point to in my sonar-project.properties? In any case, I look forward to your reply!
you should use the bw-outputs of the machine where you are analyzing, regarding the report locally generated on the same machine you shouldn’ face any issue.
For the reports generated on another machine you should make sure you relativize paths as changing the machine changes the paths.
@mpaladin An update: it seems like I got a crude example to work. However, I couldn’t replicate this on Linux. It seems when trying to run the scanner when a Visual Studio coverage XML is present, the Windows build wrapper’s bw-outputs is required. However for Linux, even if you don’t include its bw-outputs, the coverage will successfully upload.
This is the result where sonar-project.properties only pointed to a Visual Studio coverage XML report:
It would be nice if in the combined report that the green bars show up for the Windows-only section as well. However if you look at the numbers, they seem to add up correctly!
it comes from the fact that the UI shows the coverage for the lines which are marked as executable from the analyzer. Currently, the analyzer is marking as executable only the lines touched by the preprocessor during the analysis, that explains why you don’t see a green bar on the lines inside the #ifdef _WIN32 branch.
We also have the same need here, some source files are drivers for specific arm targets. Retrieving the gcov files to the main build is no problem but it’s a pitty that if the lines were not marked as executed on the main build they won’t be marked as covered. Would it be possible to add an option to bypass that check? I understand it makes the report more reliable when building and running on only one targets but it is annoying when you have multiple targets.
it is not as easy as setting an option to bypass the check. The analyzer need to know which lines are executable lines of code, we currently don’t have a way of knowing all configuration executable lines of code on the same run. I am sorry but we don’t have a solution for the time being.
