Version
Observed on SonarQube Community Build 25.2 with the C# plugin 10.6 (deployment type should not matter here).
Also reproduced against the current master of sonar-dotnet (Coverage.java) with the unit tests below.
The reports I used are in the OpenCover format, but the bug is in the format-independent merge.
What happens
When several coverage reports contain the same source file but do not list the same set of lines, the merged result depends on the order in which the reports are read:
- Lines that are executable but not covered disappear from “lines to cover”, so the coverage percentage is inflated.
- Lines covered by only one of the reports lose hits. A line hit once ends up with 0 hits.
I noticed it because the same commit, analyzed twice with the same reports, produced very different coverage percentages. The number of covered lines was the same in both runs, only the number of lines to cover changed.
Root cause
Coverage.mergeLineHits passes every slot of the other report’s array to addHits, including the slots that hold SPECIAL_HITS_NON_EXECUTABLE (-1):
for (int i = otherHitsByLine.length - 1; i >= 0; i--) {
addHits(file, i + 1, otherHitsByLine[i]);
}
addHits then adds that -1 to the existing value:
| existing | other | result | expected |
|---|---|---|---|
| 0 (uncovered) | -1 (line not in report) | -1 (not executable) | 0 |
| 1 | -1 | 0 | 1 |
| N | -1 | N - 1 | N |
| -1 | 0 or N | 0 or N | same (correct) |
So a.mergeWith(b) and b.mergeWith(a) give different results as soon as one report lists a line that the other one does not.
How to reproduce
No project is needed, three unit tests on Coverage are enough (code at the end). With the current master:
full.mergeWith(partial) -> {10=1, 11=1} expected {10=1, 11=1, 12=0, 13=0}
partial.mergeWith(full) -> {10=1, 11=1, 12=0, 13=0} correct
partial.mergeWith(other) -> {10=0, 11=0, 12=1, 13=1} expected {10=1, 11=1, 12=1, 13=1}
When does this happen in practice
Any setup where two reports describe the same file with different line sets. In my case it is the Unity Code Coverage package: for each test mode it writes two OpenCover files, one listing every sequence point with zero visits and one containing only the methods that were actually called. Both files match sonar.cs.opencover.reportsPaths, and the result depends on which one the scanner reads first.
Fix
Skip the non-executable slots when merging:
for (int i = otherHitsByLine.length - 1; i >= 0; i--) {
if (otherHitsByLine[i] != SPECIAL_HITS_NON_EXECUTABLE) {
addHits(file, i + 1, otherHitsByLine[i]);
}
}
With this change the three tests below pass, and the existing CoverageTest tests still pass.
I have opened a pull request with the fix and the tests: Fix coverage merge dropping uncovered lines when reports list different lines of the same file by jh11240 · Pull Request #9856 · SonarSource/sonar-dotnet · GitHub
Unit tests (to add to CoverageTest.java)
private static final String FILE = "Foo.cs";
@Test
public void mergeWith_otherHasFewerLines_keepsUncoveredLines() {
Coverage sut = allLinesUncovered();
sut.mergeWith(onlyFirstMethodCovered());
// Lines 12 and 13 are executable and uncovered. They must stay in the lines to cover.
assertThat(sut.hits(FILE)).isEqualTo(Map.of(10, 1, 11, 1, 12, 0, 13, 0));
}
@Test
public void mergeWith_isOrderIndependent() {
Coverage fullFirst = allLinesUncovered();
fullFirst.mergeWith(onlyFirstMethodCovered());
Coverage partialFirst = onlyFirstMethodCovered();
partialFirst.mergeWith(allLinesUncovered());
assertThat(fullFirst.hits(FILE)).isEqualTo(partialFirst.hits(FILE));
}
@Test
public void mergeWith_disjointLines_keepsHits() {
Coverage sut = onlyFirstMethodCovered();
sut.mergeWith(onlySecondMethodCovered());
// Each report covers a different method. No hit may be lost.
assertThat(sut.hits(FILE)).isEqualTo(Map.of(10, 1, 11, 1, 12, 1, 13, 1));
}
// A report listing every sequence point of the file (lines 10 to 13), none of them executed.
private static Coverage allLinesUncovered() {
Coverage coverage = new Coverage();
for (int line = 10; line <= 13; line++) {
coverage.addHits(FILE, line, 0);
}
return coverage;
}
// A report listing only the first method (lines 10 and 11), executed once.
private static Coverage onlyFirstMethodCovered() {
Coverage coverage = new Coverage();
coverage.addHits(FILE, 10, 1);
coverage.addHits(FILE, 11, 1);
return coverage;
}
// A report listing only the second method (lines 12 and 13), executed once.
private static Coverage onlySecondMethodCovered() {
Coverage coverage = new Coverage();
coverage.addHits(FILE, 12, 1);
coverage.addHits(FILE, 13, 1);
return coverage;
}