Hi, I have a pipeline where I introduced, according to instructions at
.NET Test Coverage | SonarQube Docs chapter “Extension for Azure Devops” it is indicated
arguments: ‘–collect “Code Coverage”’ # This is all you need to add!
in the test stage
Pipeline configured as follows
here is the test step
jobs run successfully
but nothing is shown on the sonarqube project page regarding coverage not test units reports
Am I missing something?
What do the logs say?
I have visited the link SonarScanner for Azure DevOps
but nowhere it is indicated where I can find the log files
I have setup the scanner using on the build agent
dotnet tool install --global dotnet-sonarscanner
and the following extension is setup on AzureDevops Server
SonarQube - Visual Studio Marketplace
OK, got it I thought you were talking about other logs. Anyway, tests are run correctly, coverage repor is created as you can notice in the snapshot , even I added “Publish code coverage from”
Also, I found this error in the “Run Code Analysis” step
in spite of the fact that dotnet-cover is setup and when I run it on command line it works, of course on the building agent and not on the sonarQube server? Should this be running on the soanrqube server !!!
Hey there.
The scanner does require Microsoft.CodeCoverage.Console tool to be available on the machine performing analysis in order to convert/merge test reports.
Are you using self-hosted build agents? What version of Visual Studio is installed, and what edition?
Yes, I use self-hosted which is on a win 10 physical machine. I have Visual Studio Professional, I am aware of the fact that coverage feature is in Enterprise edition, however, that is why when I read instructions at .NET Test Coverage | SonarQube Docs
I understood that this can allow to not need Enterprise edition and that is why it is installed globally? Non?
Colin, after a long discussion on the github project for Coverlet at
petli very nice guy and really helped a lot. After a succesful run of the build pipeline, were created correctly and I was able to see nice reports inside Azure devops, I switched to sonarQube UI, still I see that sonarqube gets alerted of the new build but does not display coverage and unt test details. After investigating, I discovered that Azure devops creates coverage reports in the _temp directory of the build agent and not the project working directory which seems that sonarQube task does not search this upper directory (I came this thread which confimed this)across this thread
So the question : where in the chain or project we can tell sonarQube to search other directories for coverage code?
Thanks in advance
I felt a pull in The Force, as if someone had said my name. =D
I dont really understand this response!!!
You pasted a screenshot of a github issue I had reported in this issue. I was not tagged or mentioned.
Less than 5 hours later I find myself looking at this thread, Probably just coincidence.
Or maybe I should ask the question differently, Is there a a parameter/Config in Sonarqube extension for Azure devops to tell it what is the location of the reports?
Hi,
If I understood correctly you dont have Visual Studio Enterprise version in your self-hosted build agent, so you need to use dotnet-coverage tool (for example). The dotnet-coverage will convert .converage format to .xml format and you need to tell sonarscanner where is the xml file. See sonar.cs.vscoveragexml.reportsPaths
in Test Coverage Parameters | SonarQube Docs
So in your SonarQubePrepare-task you will need to add in extraProperties following:
sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)/myfile.coverage.xml (for the path see the -o parameter in bash script below)
In your Test-task either add following argument
--results-directory $(Agent.TempDirectory) (or any folder you want) or do that in the .runsettings file
The .coverage files are seperated by the test cases and need to be merged and converted to one xml-file, so after Test-task add following bash script task to merge (-r argument) and convert (-f argument) .coverage files
- bash: dotnet-coverage merge -o $(Agent.TempDirectory)/myfile.coverage.xml -f xml -r $(Agent.TempDirectory)/*.coverage
displayName: 'Convert coverages to xml'
hope this helped =)
Edit: maybe merging to one xml file is not mandatory (so you could have multiple xml files) but that is what I ended with
so many thanks Aleski, I tried something likethis
sonar.projectBaseDir=$(Agent.TempDirectory)**\coverage.cobertura.xml(this is the way ** that azure devops reference su-directoris as the ml file is generated in temporary subdirectories and not directly in the _temp directoy)
in the prepare step but sonar gives an error, 2nd snapshot is for the error, indicating illegal carater, I tied to enclose with double quotes but also refuses
as well as other test but they all fail;ed because of the 2 or 1 asterk
Is there a possiblity o make understand that ** is meant should look in subdirectories?
Because it shoud be inerpreted as
C:\Projects\agent-
AzureDevOps2K20Update15_work_temp\2321254\coverage\coverage.cobertura.xml
The 2321254 directory is reatredon the fly dynaically by the bild process
You are using wrong config value in you prepare analysis step.
so change sonar.projectBaseDir → sonar.cs.vscoveragexml.reportsPaths
and I’m not sure does SonarQube support cobertura xml format for C#, that is why I’m using xml format and that sonar.cs.vscoveragexml.reportsPaths config value.
so cobertura xml is different than (vscoverage)xml imho. See for example
dotnet-coverage code coverage tool - .NET CLI - .NET | Microsoft Learn
Here is the structure of the _temp directory during uild process
as soon as I add --results-directory to test step the test step it fails.
so I removed
I replaced sonar.projectBaseDir by sonar.cs.vscoveragexml.reportsPaths and ran the build pipeline
Ok, after removing results directory from test step and keeping sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)**\coverage.cobertura.xml
in prepare step, build goes through but it seems there is an issue sonar acceptig self-igne certificate as follows
and thisis the error on sonar server side, is it possible to tellsonar not to validate certificate? however my certifdicate is accepted b google and edge without any issues
Regarding the format, MS guys on community confirmed that I should use
–collect:“XPlat Code Coverage”
I guess they didn’t. They just gave you an example runsettings file where was XPlat Code Coverage specified and the last comment was
“The issues you have now is something with how your build system runs your tasks, and I can’t assist with that. For sonarqube you have to read their documentation on how to specify the coverage report path. You may find https://reportgenerator.io/ useful if you need to convert between report formats or if you have multiple coverlet output files that must be merged into a single report.”
But now I remember that I also had problems with those DotNetCoreTasks which you are using (I’m using YAML pipelines anyways and not the classic UI defined pipelines).
So the Test-task actually incjects to --logger trx and --results-directory arguments to the dotnet test command what you see in your logs it can be seen from Info icon next to ‘Publish test results and code coverage’
so if you use that the reports are always going to $(Agent.TempDirectory).
I would recommend to change back to --collect "Code coverage" setting, adding the bash script task with following command
dotnet-coverage merge -o $(Agent.TempDirectory)/myfile.coverage.xml -f xml -r $(Agent.TempDirectory)/*.coverage
and setting in prepare analysis
sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)/myfile.coverage.xml (NOTE: specific file that we do with bash script)
For the certificate error in Publish Quality Gate results you need to add correct certificate chain to your self-hosted build agent’s Trusted Certs.
Hope this helps you, have a nice weekend!
Ok I will update the build pipeline with your suggestion but can you tell me how it will find
myfile.coverage.xml
which is not directly under $(Agent.TempDirectory)? it is a lower level directory created on the fly as the screenshot I posted earlier
