C#/VB.NET: SonarQube and SonarCloud support Branch/Condition Coverage data

Dear DotNet Developers,

We are happy to announce that we have updated our support of DotNet code coverage in both SonarQube and SonarCloud. You will now be able to appreciate a more consistent code coverage, with a calculation based on line coverage AND branch coverage.

SonarQube and SonarCloud both help you:

  • to control your application’s coverage level thanks to the Quality Gate
  • to visualize coverage data and determine where you should focus to improve it.

Here is how it works: The overall Coverage metric is computed using Line Coverage (LC) and Branch Coverage data provided by the various DotNet coverage tools available on the market (OpenCover, Coverlet, NCover and VSTest) like this:

Coverage = (CT + CF + LC)/(2*B + EL)

where

  • CT = conditions that have been evaluated to ‘true’ at least once
  • CF = conditions that have been evaluated to ‘false’ at least once
  • LC = covered lines = linestocover - uncovered_lines
  • B = total number of conditions
  • EL = total number of executable lines (lines_to_cover)

Branch/Condition Coverage

With the great feedback some of you gave us, we realized that what was displayed on the various coverage tools (OpenCover, Coverlet, NCover and VSTest) was not consistent with SonarQube/SonarCloud.

This was linked to the fact we were considering only Line Coverage data and not Branch Coverage ones. The impact is that a line containing a conditional statement is fully considered as covered whereas some of the condition’s branches are not fully covered.

In the following example, you can see that some “getter” and “setter” are not covered at all (red underlying) and the condition at line 17 is partially covered (yellow underlying). This is what is reported by Visual Studio (VSTest):

In the past, SonarQube / SonarCloud were considering incorrectly everything as fully covered:

To avoid this approximation, we looked at the data provided by the various tools and found that:

  • OpenCover offers branch coverage data
  • Coverlet offers branch coverage data in OpenCover format starting from v1.7.44 and the contribution we made
  • VSTest doesn’t support precise branch coverage in its exported report
  • dotCover doesn’t support branch coverage data

As a result, here are the OpenCover data loaded into SonarQube with the new branch coverage support:

With this change, SonarQube/SonarCloud are very close to what is reported in Visual Studio and OpenCover itself.

Expected Impacts

You can expect two impacts from this change:

  • the coverage data displayed and computed (Coverage, Conditions Coverage metrics) in SonarQube/SonarCloud are closer to reality
  • a drop of your overall coverage: a line that was considered as fully covered will now be accurately considered as partially covered if some conditions are not covered

This change will be deployed in the coming days on SonarCloud, and will be included in SonarQube 8.3.

We thank you again for your feedback that helped us improve our products, and we encourage you to let us know how that works for you!

Enjoy!

5 Likes

Is there a way to disable this feature? Our coverage just dropped into oblivion and there is no real way for us to go back on our old code o bump up the condition coverage.

No, you can’t deactivate this behavior. Today, it’s more accurate than before so being able to deactivate the feature would be just a way to sweep the dust under the carpet.
I believe you should not worry too much about the old code not so well covered but make sure that the new code is well covered so adopt the Clean/Cover as You Code approach.

yeaaaah you’re right, but in our position, we thought it was REALLY weird that a build for a PR started being rejected over night. was this announced before? no hard feelings, just wondering.

A post was split to a new topic: Dotnet setter code coverage in serialization scenarios

A post was split to a new topic: Have different highlight within the file on what covered by the conditions in sonarcloud

A post was split to a new topic: Getter/setter properties said not covered with C# analyzer 8.6

2 posts were split to a new topic: C# branch code coverage in SonarCloud

We’ve released sonar-dotnet 8.6.1, fixing the branch coverage import bug when importing data from multiple test projects.

It will get deployed to :sonarcloud: in the following days. You should expect branch (conditional) code coverage to increase.

Thank you for your feedback!

1 Like