Dear DotNet Developers,
We are happy to announce that we have updated our support of DotNet code coverage in both SonarQube and SonarCloud. You will now be able to appreciate a more consistent code coverage, with a calculation based on line coverage AND branch coverage.
SonarQube and SonarCloud both help you:
- to control your application’s coverage level thanks to the Quality Gate
- to visualize coverage data and determine where you should focus to improve it.
Here is how it works: The overall Coverage metric is computed using Line Coverage (LC) and Branch Coverage data provided by the various DotNet coverage tools available on the market (OpenCover, Coverlet, NCover and VSTest) like this:
Coverage = (CT + CF + LC)/(2*B + EL)
- CT = conditions that have been evaluated to ‘true’ at least once
- CF = conditions that have been evaluated to ‘false’ at least once
- LC = covered lines = linestocover - uncovered_lines
- B = total number of conditions
- EL = total number of executable lines (lines_to_cover)
With the great feedback some of you gave us, we realized that what was displayed on the various coverage tools (OpenCover, Coverlet, NCover and VSTest) was not consistent with SonarQube/SonarCloud.
This was linked to the fact we were considering only Line Coverage data and not Branch Coverage ones. The impact is that a line containing a conditional statement is fully considered as covered whereas some of the condition’s branches are not fully covered.
In the following example, you can see that some “getter” and “setter” are not covered at all (red underlying) and the condition at line 17 is partially covered (yellow underlying). This is what is reported by Visual Studio (VSTest):
In the past, SonarQube / SonarCloud were considering incorrectly everything as fully covered:
To avoid this approximation, we looked at the data provided by the various tools and found that:
- OpenCover offers branch coverage data
- Coverlet offers branch coverage data in OpenCover format starting from v1.7.44 and the contribution we made
- VSTest doesn’t support precise branch coverage in its exported report
- dotCover doesn’t support branch coverage data
As a result, here are the OpenCover data loaded into SonarQube with the new branch coverage support:
With this change, SonarQube/SonarCloud are very close to what is reported in Visual Studio and OpenCover itself.
You can expect two impacts from this change:
- the coverage data displayed and computed (Coverage, Conditions Coverage metrics) in SonarQube/SonarCloud are closer to reality
- a drop of your overall coverage: a line that was considered as fully covered will now be accurately considered as partially covered if some conditions are not covered
We thank you again for your feedback that helped us improve our products, and we encourage you to let us know how that works for you!