Condition coverage not shown for `coverage.txt` generated by `llvm-cov show`

Hi,

I don’t see that coverage report generated by llvm-cov show contains any information related to condition coverage.

Here is a screenshot:

We generate report as follows:

llvm-cov show tests-coverage.x -instr-profile=tests/temp/tests.profdata -show-expansions -show-regions -show-line-counts -use-color=false > coverage.txt

Here is part of coverage.txt which includes region coverage information:

   34|  8.98k|    if (_isWaitingAuthentication && package->requiresAuthentication()) {
                                                  ^3.24k

Could you help with this?

Thanks!

Hi @manuyavuz-pointr,

I don’t really understand what is your problem and what you would like to achieve.

Hi @mpaladin,

We would like to see condition coverage in our dashboard, and currently it’s not shown.

It only shows line coverage. However, I have verified that coverage.txt file includes information about condition coverage (region coverage in llvm-cov terminology).

Here is result for llvm-cov report command for example:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Filename                                                            Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover
TOTAL                                                                 15695              7027    55.23%        2944               915    68.92%       29242              9523    67.43%

Regions here correspond to Conditions in sonar terminology. We expect to see this to properly presented in sonarcloud dashboard.

Thanks

Hi @manuyavuz-pointr,

I confirm that currently we don’t support import of condition coverage for llvm cov format. Are using llvm cov on a linux project or on macos xcode project?

We use it in our makefile based cpp project which can be run both on linux and macos.

I am trying to use llvm-cov gcov to get gcov based report. But it’s still not clear to me how it performs name matching between generated .gcno and .gdca files and given source files. It looks for wrong path for generated coverage files.

What’s your suggested way to generate report in this case? It would have been great if we could directly use llvm-cov’s default coverage format, not gcov one.

Also, I suggest you to update your documents on test coverage regarding this limitation.

https://docs.sonarqube.org/latest/analysis/coverage/

Above guide does not include anything about condition coverage limitation of llvm-cov.

Hi @manuyavuz-pointr,

if you want condition coverage it would be indeed faster to generate gcov on your side.

I created a ticket to add support for condition coverage but no ETA and you are the first to ask about it: CPP-2340.

@mpaladin, when I try gcov report, I get the following result from sonar-scanner:

12:24:08.369 DEBUG: 'gcov' skipped because one of the required properties is missing

This is what I have as sonar-scanner parameter:

-Dsonar.cfamily.gcov.reportPath=.

And I have .gcov files under project base dir in native gcov format generated by llvm-cov gcov.

What could be the problem?

Unfortunately I can’t paste full sonar-scanner output as it’s a private project. How do you proceed to support private projects for these cases?

Best

Hi @manuyavuz-pointr,

The property is sonar.cfamily.gcov.reportsPath (note the typo).

Hi @mpaladin,

Thanks so much, and sorry for taking your time for such a careless mistake.

I could collect branch coverage results form gcov reports now, but there is a problem with that, which is explained here in detail: https://stackoverflow.com/questions/42003783/lcov-gcov-branch-coverage-with-c-producing-branches-all-over-the-place

Basically, gcov treats compiler generated extra blocks (such as try catch blocks) for some cases as branches too, and assumes partial coverage for some weird code blocks such as the following:

## gcov sees below line as having 2 branches, 1 covered
static std::string versionString = "Unknown";

## gcov sees below line as having 4 branches, 3 covered
for (auto ball : balls) {
}

Do you know any workarounds for these type of issues?

Calculating condition coverage result this way is not really helpful for code coverage assessment. For reference, condition coverage shown by sonarcloud for gcov coverage report is 37%, whereas for llvm-cov's default format it’s around 58%.

Due to above limitations, it’s really an urgency for us to be able to use llvm-cov's native coverage format to get correct condition coverage percentages.

Hi @manuyavuz-pointr,

The only thing I can think of is to generate gcov format from the start without passing by llvm-cov format.

I cannot promise anything on this side, we will try to give a look.

The only thing I can think of is to generate gcov format from the start without passing by llvm-cov format.

I don’t think it’s related to llvm-cov because the stackoverflow link that I sent uses gcc's native gcov tool, not llvm-cov.

Seems like we will have to revert back to native llvm-cov format, and wait for your side to implement the feature.

Is there any way to watch the corresponding JIRA task?

Thanks for your support!

Hi @manuyavuz-pointr,

I believe you should be able to watch the ticket if you register/login to jira.

Cool, I’m in!