We have got a .NET Core 3.0 solution with 4 (business logic) projects and 5 unit testing projects. These test projects each cover one specific part of the code. We use Azure DevOps pipelines to deploy and test the solution and report the code coverage the Sonar Source. For reporting the code coverage we use the
OpenCover format. After the recent update on the code coverage (which is much better than just line coverage) our coverage dropped from 90% to about 60% which seems like a lot.
When we took a look at the reported code coverage in Azure DevOps we noticed that it was completely different from the coverage reported in SonarSource. SonarSource seems to report way too many conditions. For instance, a simple
if(condition) would result in 10 possible conditions which clearly is incorrect, where Azure DevOps just reports 2 possible conditions.
The problem appears to be that for every test project an OpenCover file is created which shows the coverage for that single test projects. Every OpenCover file contains all statements and their conditions. SonarSource properly merges line coverage, but it appears to sum the amount of conditions. So where the most simple if statement should have 2 conditions, SonarSource actually reported 2 x 5 = 10 conditions (we have got 5 test projects) with only 2 conditions being covered.
We use the settings below to generate the OpenCover files.
SonarCloud prepare task
- task: SonarCloudPrepare@1 inputs: SonarCloud: 'MySonarCloud' organization: 'MyOrg' scannerMode: 'MSBuild' projectKey: 'SomeProjectKey' projectName: 'MyProject' extraProperties: | sonar.cs.vstest.reportsPaths=$(TestOutputDirectory)/*.trx sonar.cs.opencover.reportsPaths=$(TestOutputDirectory)/*/coverage.opencover.xml sonar.coverage.exclusions=**/Migrations/*.cs,**/*Tests*/**/* sonar.verbose=true
Dotnet Test task
- task: DotNetCoreCLI@2 displayName: 'dotnet test' inputs: command: test publishTestResults: false projects: '$(project)' arguments: '--no-restore --no-build --configuration $(BuildConfiguration) --logger trx --collect:"XPlat Code Coverage" --results-directory $(TestOutputDirectory) --settings $(Build.SourcesDirectory)/coverlet.runsettings'
$(project) points to our solution file. We also tried to test each project individually but the result was exactly the same.
We use ReportGenerator to report the code coverage in Azure DevOps.
- task: reportgenerator@4 inputs: reports: '$(TestOutputDirectory)/*/coverage.opencover.xml' targetdir: '$(TestOutputDirectory)/mergedcoveragereport' reporttypes: 'HtmlInline_AzurePipelines;Cobertura' assemblyfilters: '-*Tests*' filefilters: '-*/Migrations/*.cs'
To fix the problem we change the
SonarCloudPrepare task to use the
coverageReportPaths instead of
OpenCover and we added
SonarQube to the reporttypes created by the
reportgenerator tasks. Now SonarSource is using the
SonarQube.xml file to report the code coverage and we have got it back to about 90%.