SonarQube incorrectly aggregates SimpleCov Ruby coverage reports

For one of our Ruby projects, we split its test suite into several chunks, run them in parallel, and feed SonarQube with all partial code coverage reports. That, however, results in SonarQube incorrectly reporting non-executable lines of code (empty, ends, etc.) as not covered.

This is caused by SonarQube incorrectly merging several SimpleCov reports. For example, for one of our classes, SimpleCov generates the report below (shortened for example purposes) for the test chunk that includes its tests:

"/w/some/file.rb": {
  "lines": [
    1,
    2,
    null,
    1,
    null,
    null
  ]
},

test chunks that don’t include that class’ tests result in a report like

"/w/some/file.rb": {
  "lines": [
    0,
    0,
    0,
    0,
    0,
    0
  ]
},

which SonarQube logic aggregates into

"/w/some/file.rb": {
  "lines": [
    1,
    2,
    0,
    1,
    0,
    0
  ]
},

Resulting in 50% line coverage for a file, where the real coverage is 100%.

SimpleCov itself also includes aggregation logic, and that logic explicitly says that null and 0 should be merged into null : simplecov/lib/simplecov/combine/lines_combiner.rb:

# Return depends on coverage in a specific line
#
# @param [Integer || nil] first_val
# @param [Integer || nil] second_val
#
# Logic:
#
# => nil + 0 = nil
# => nil + nil = nil
# => int + int = int
#
# @return [Integer || nil]

I’m open to submit a fix for the logic, but would love for someone to confirm my findings first.

Related posts:

1 Like

Hey there.

What version of SonarQube are you using?

Hey @Colin! We’re using SonarQube 10.4.1.

Is this the same issue you reported here?

I’m asking because I already pinged the relevant team on that thread and their response is pending. Just don’t want to duplicate effort!

You’re right, that’s the same issue. I’m sorry for reposting it. As I dug into that issue, I believe I’ve got a fix for it that I can contribute, but I’d like someone to confirm if that’s ok and if so, create a ticket for that.