Azure Devops .net core 3.1 not generating coverage

Hi,

currently we have our builds on running on Azure DevOps on Windows agents and everything works.

We tried now to port it to a Linux (Ubuntu) agents. Unfortunately the build doesn’t calculate code coverage one the Ubuntu machine. It constantly stays empty and therefore the build breaks on every attempt.

I tried several combinations of different yaml files and googled a lot but couldn’t resolve the issue. My current yaml looks like this.

- task: SonarCloudPrepare@1
    displayName: 'Prepare analysis on SonarCloud'
    inputs:
      SonarCloud: 'xxxx'
      organization: 'xxxxx'      
      projectKey: '$(SonarProjectName)'      
      projectName: '$(Build.Repository.Name)'
      extraProperties: | 
        sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/TestResults/Coverage/Reports/coverage.opencover.xml

- task: DotNetCoreCLI@2
    displayName: 'Install dotnet-reportgenerator-globaltool'
    inputs:
      command: custom
      custom: tool
      arguments: 'install --global dotnet-reportgenerator-globaltool --ignore-failed-sources'

- task: DotNetCoreCLI@2
    displayName: 'dotnet test'
    inputs:
      command: 'test'
      projects: '**/*[Tt]est*(s)/*/*.csproj'
      # arguments: '--configuration $(BuildConfiguration) --collect "XPlat Code Coverage"'      
      arguments: '--configuration $(BuildConfiguration) --collect "XPlat Code Coverage" --logger "trx;LogFileName=test-results.trx" /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/'
      nobuild: true

  - script: |
      echo "##vso[task.prependpath]$HOME/.dotnet/tools"      
      reportgenerator "-reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml" "-targetDir:$(Build.SourcesDirectory)/TestResults/Coverage/Reports" -tag:$(Build.BuildNumber) -reportTypes:htmlInline              
    displayName: Create Code coverage report

  - task: PublishTestResults@2
    inputs:
      testRunner: VSTest  
      testResultsFiles: '**/*.trx'
      failTaskOnFailedTests: true

  - task: PublishCodeCoverageResults@1
    displayName: 'Publish code coverage'
    inputs:
      codeCoverageTool: Cobertura
      summaryFileLocation: $(Build.SourcesDirectory)/TestResults/Coverage/**/coverage.cobertura.xml
      reportDirectory: $(Build.SourcesDirectory)/TestResults/Coverage/Reports
      failIfCoverageEmpty: false

This is my first post in the forum, please let me know if you need any additional info.

Any help is greatly appreciated.

Thanks in advance

  • ALM used Azure DevOps
  • CI system Azure DevOps
  • Languages of the repository: .net core 3.1
  • Error observed: Code coverage not calculated in Sonar

Hi @dre and welcome to the community !

Is that possible for you to share the log of the Run Code Analysis task in debug mode please ?

Thanks.

Hi @mickaelcaro,

thanks for the quick response. Here the log.

https://pastebin.com/raw/mnFLz2ZY - Log 1
https://pastebin.com/raw/xJ3HQBQG - Log 2
https://pastebin.com/raw/kHfTGe6z - Log 3
https://pastebin.com/raw/Vv78ev20 - Log 4

Sorry for the multi-part file. It was too big to upload it in one go.

Hi again,

It appears that the file you provide (and the path) was not found. Could you please check that it’s here ?

Could not find any coverage report file matching the pattern ‘/home/vsts/work/1/s/TestResults/Coverage/Reports/coverage.opencover.xml’.

Thanks !

Hi @mickaelcaro,

thanks for the hint. The file is not generated as far as I see. I tried a few different configurations last week but none of them seem to work and generate the coverage. Do you have maybe a demo configuration for .net core 3.1 on Linux agents? Is there something wrong with the parameters that I am using?

Thanks

I was wondering if that may work since your are using coverlet, but as far as i know, coverage is still not supported on linux agent, see this thread : https://github.com/Microsoft/vstest/issues/981

Mickaël

Thanks @mickaelcaro. I was skimming through the github thread and it seems to be an ongoing issue. Is there any alternative way instead of coverlet to get it working with Sonarcloud on a Linux agent?

I found that : https://automationrhapsody.com/net-core-code-coverage-linux-minicover/
And that : https://dotnetthoughts.net/code-coverage-in-netcore-with-coverlet/

But haven’t tested them.

Would you have a look ?

This was actually a push in the right direction. Now I am not getting any more warnings from the tasks. I switched to opencover but still experience the same issue. I was following the second tutorial that you posted and filled the gaps with this one:https://writeabout.net/2019/04/27/net-core-code-coverage-done-right/

Now I see in Azure Devops, the correct coverage calculated but it is not propagated to Sonar.

  - task: SonarCloudPrepare@1
    displayName: 'Prepare analysis on SonarCloud'
    inputs:
      SonarCloud: 'xxxxxx'
      organization: 'xxxxxx'      
      projectKey: '$(SonarProjectName)'      
      projectName: '$(Build.Repository.Name)'
      extraProperties: |         
        extraProperties: 'sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/TestResults/Coverage/Reports/coverage.opencover.xml'
        sonar.verbose=true


  - task: DotNetCoreCLI@2
    displayName: 'dotnet test'
    inputs:
      command: 'test'
      projects: '**/*[Tt]est*(s)/*/*.csproj'      
      arguments: '--configuration $(BuildConfiguration) --logger "trx" /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/Reports/'
      nobuild: true

  - script: |
      # echo "##vso[task.prependpath]$HOME/.dotnet/tools"      
      # reportgenerator "-reports:$(Build.SourcesDirectory)/TestResults/**/coverage.cobertura.xml" "-targetDir:$(Build.SourcesDirectory)/TestResults/Coverage/Reports" -tag:$(Build.BuildNumber) -reportTypes:htmlInline
      reportgenerator "-reports:$(Build.SourcesDirectory)/TestResults/**/coverage.opencover.xml" "-targetDir:$(Build.SourcesDirectory)/TestResults/Coverage/Reports" -tag:$(Build.BuildNumber) "-reporttypes:Cobertura;HTMLInline;HTMLChart"
    displayName: Create Code coverage report

  - task: PublishTestResults@2
    inputs:
      testRunner: VSTest  
      testResultsFiles: '$(Agent.TempDirectory)/**/*.trx'
      failTaskOnFailedTests: true

  - task: PublishCodeCoverageResults@1
    displayName: 'Publish code coverage'
    inputs:
      codeCoverageTool: Cobertura
      summaryFileLocation: $(Build.SourcesDirectory)/TestResults/Coverage/Reports/Cobertura.xml
      reportDirectory: $(Build.SourcesDirectory)/TestResults/Coverage/Reports
      failIfCoverageEmpty: false 
      
  - task: SonarCloudAnalyze@1

  - task: SonarCloudPublish@1
    inputs:
      pollingTimeoutSec: '300'

  - task: sonarcloud-buildbreaker@2
    inputs:
      SonarCloud: 'xxxxxx'
      organization: 'xxxxxx'

Thanks

When you run the SonarCloudAnalyze task, in debug mode, do you see some kind of debug log around the coverage processing ? It should tell you how many files it discovered and so on… Would be probably helping you troubleshoot.

Sorry, I forgot to append the logs in the last reply. I am not sure what I am looking for in the logs.
I noticed just that opencover is just mentioned twice in the upper section of the log so I am not sure if Sonar picks it up.

https://pastebin.com/tVY0Cchu - partI
https://pastebin.com/QGngsC78 - partII
https://pastebin.com/AnTWy2FC - partIII
https://pastebin.com/yVYizZeR - part IV

Thanks

I think there’s a typo here, extraproperties shouldn’t be put twice, you only need something like this :

      extraProperties: | 
sonar.cs.opencover.reportsPaths=$(Build.SourcesDirectory)/TestResults/Coverage/Reports/coverage.opencover.xml

I figured it out now, there are were a few things I had to change but you pushed me in the right direction with the your last answer.

  1. Change the path in the extraproperties
  2. Adding the coverlet.msbuild package to every .csproj file
<PackageReference Include="coverlet.msbuild" Version="2.9.0">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
  1. Remove coverlet output from .net test task so the file is stored in the same directory as the project file
  2. Change the report generator task -reports parameter "-reports:$(Build.SourcesDirectory)/**/coverage.opencover.xml" to search in all directories
  3. Run .net test on the .sln file instead of single project files

This helped me to fix it. Now coverage is generated and uploaded the Sonarcloud. I also added attributes so that the ExcludeFromCodeCoverage attribute is taken into account.

/p:ExcludeByAttribute=ExcludeFromCodeCoverage /p:ExcludeByAttribute=ExcludeFromCodeCoverageAttribute

There is just one thing I noticed that is weird. The coverage with open cover is about 5 percent lower when running on the windows agent. Event though the report shows the same line coverage. Do you have any idea on top of your mind?

Windows agent
image

Linux agent
image

Report from Linux build
image

Thanks @mickaelcaro for all the help!

Hi !

That’s a great news ! :slight_smile:

Well i don’t know, do you have the possibility to open both coverage files in Visual Studio to see where the coverage is missing ? So then you’ll have a clue.

Mickaël

1 Like

Good advice, some of the test projects were included in the windows build definition so now it will be even more accurate :slight_smile: Thanks once more @mickaelcaro you really helped me a lot!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.