Unexpected coverage when supplying comma-separated LCOVs

SQ Enterprise Edition Version 8.5 (build 37579)
SonarScanner 4.5.0.2216

Our project’s front-end unit tests produce multiple LCOV files, as shared packages will have their very own unit tests and reporters. I’m supplying these files to “sonar.javascript.lcov.reportPaths” as comma-separated values and for each LCOV I’ve gone ahead and corrected all of the paths that are found inside (“SF:path/to/jsFile”) to be relative to the git repository’s root (also the base directory of my project within SonarQube). With the file paths corrected, the LCOVS appeared to process correctly.

The issue is that there’s some overlap in covered files.

If I set sonar.javascript.lcov.reportPaths to include only one LCOV file (let’s call it lcov_A), the SQ dashboard reports a coverage of 81% for a particular file (let’s call it dashboard.js).

If I set sonar.javascript.lcov.reportPaths to include a different LCOV file (still only one, and let’s call it lcov_B), the SQ dashboard reports a coverage of 4% for that same file, dashboard.js.

So far so good. Nothing unexpected.

Now if I set sonar.javascript.lcov.reportPaths to be a comma-separated list of lcov_A and lcov_B, the SQ dashboard reports a coverage of 65%. So my big question is: where did this number come from???

For the vast majority of JS files, SonarQube seems to list the maximum value from lcov_a and lcov_B for a particular file (using the previous example, it would have been 81%). But for a small subset of JS files, SonarQube reports a value that’s somewhere in-between the lcov_A and lcov_B values for the file. How exactly is this being calculated? Is it a bug?

I would find either of the following scenarios to be acceptable:
A) the max coverage value for a given JS file is used (whichever LCOV reports the highest value for that particular file gets used)
B) the lcovs are merged by SQ, which could potentially result in something that’s higher than both values (e.g., perhaps it considers lines 5-10 as covered by lcov_A and lines 15-25 by lcov_B, which means the merged lcov has a higher percentage than either of original LCOVs on their own).

But what I was NOT expecting is a value somewhere in-between. It’s not an average either. What is it? It doesn’t seem to occur on every file either. I don’t have a concrete example to share just yet. I’m hoping that perhaps someone else may have encountered this issue as well.

Thanks in advance to anyone willing to assist!

Hi @jburr,

Welcome to SonarSource community! :wave:

Please read the metric definition of coverage from our documentation.

You are most likely looking at coverage metric. In essence, coverage is a mix of line coverage and condition coverage:

Coverage (coverage)
It is a mix of Line coverage and Condition coverage. Its goal is to provide an even more accurate answer to the following question: How much of the source code has been covered by the unit tests?

Coverage = (CT + CF + LC)/(2*B + EL)
where

CT = conditions that have been evaluated to 'true' at least once
CF = conditions that have been evaluated to 'false' at least once
LC = covered lines = lines_to_cover - uncovered_lines
B = total number of conditions
EL = total number of executable lines (`lines_to_cover`)

Based on how your unit tests are covering your lines of code (you can check what’s covered by the green vs. red gutter bars in the Measures->Coverage view), your coverage values will differ when combining multiple LCOV files.

Joe

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.