The Test Coverage Results Displayed in SonarCloud are Wrong
versions used : I am using SonarCloud
error observed: The Test Coverage Results Displayed in SonarCloud are Wrong
steps to reproduce
Run Analyze Test Coverage for all tests in Visual Studio
Run Sonar Cloud on the same Branch
Results: The Sonar Cloud Test Coverage Result is 2.41% and the Visual Studio Enterprise Edition Test Coverage Result is 14.86%
So, what do we know so far:
VS.Net Counts the number of lines differently then sonar cloud.
Sonar Cloud uses its line count and the line coverage in each record from the vs.net report.
On the develop branch, the resulting percentage from sonar cloud and vs.net are different by about a factor of 4, which is much greater than the factor of line count differences, which is less then 2. This means that we are missing something. My guess is that some of records from the vs.net report are not counted.
The configurations were wrong for doing code coverage on pr’s until yesterday – I do not know if that is relevant for the develop branch itself.
Our Coverage measure is a mix of the “Line Coverage %” and “Conditions Coverage %”, but you can also check those two measures separately in the “Measures” tab of your branch to check if those match what you see in Visual Studio.
You also probably want to make sure you narrowed the focus of your analysis in the same as what you see in Visual Studio, if different files are taken into account, for sure the measure will not match.
What’s showing under “Tests” depends on which language your are analyzing (not all support this metric) and if your sonar.tests property is configured correctly (might be setup automatically if you use Maven or Gradle).
And it unrelated to the coverage measures. The coverage measure are filled up thanks to the coverage report that your passing to the scanner at the time of analyse. So you should check what’s in this file and if it matches the result of Visual Studio. If it doesn’t then you probably need to tweak the parameters of the tool you are using to generate the coverage report.
