We’ve been successfully using the three tasks (SonarCloudPrepare, SonarCloudAnalyze and SonarCloudPublishQualityGate) you provide for quality assessment (with dotnet test to generate test coverage reports and running our unit tests) for a few months now. Last week code coverage started failing, both for our onprem and cloud solutions. I’m transforming our Azure DevOps build pipelines from classic to yaml, and today I discovered new versions of the Sonar tasks (which I can’t recall seeing before) and updated them in the classic pipelines. Updating those tasks partly resolved my issue.
The classic pipelines now have code coverage, both in SonarCloud and in the extension-tab on the pipeline. So I tried exporting the yaml from the classic pipeline, and wouldn’t you know? It worked! That is, until I pulled the latest commits from our main branch (which the classic pipeline handles withour errors). Then the Sonar analysis report on the extension tab and SonarCloud lacked information about test coverage.
I find it very odd, as the pipeline steps in both cases should be equal, and I can’t see that the updated code should hinder code coverage in Sonar (the updated code is just nuget package updates, which, as I’ve already’ve mentioned, hasn’t prevented code coverage for the classic pipeline). The other tabs in the pipeline shows code coverage and test results correctly, so I’m trying to find out why quality gate behaves this way. Any ideas? I see that the information in the extension tab stems from the build’s artifactstaging-directory in a markdown file, but I can’t find how that file or other data passed to sonarcloud’s quality gate.
ALM used (Azure DevOps)
CI system used (Azure DevOps
Scanner command used when applicable (private details masked): MSBuild
Was redirected from the other thread, so reposting here:
To give further info:
Until a few weeks ago, we had around 10 pipelines for different system. These pipelines use the classic pipeline from Azure DevOps. But we have a goal to replace these classic pipelines with yaml pipelines, meaning the pipelines should produce the same results. However, once I fixed the code coverage issue for the classic pipelines, I haven’t fully been able to do so with the yaml pipelines (yet). This is what I’ve observed so far:
Pipeline name
Description
Result
Source code status
classic
Original classic pipeline
Code coverage is displayed both in Azure Devops (Extensions-tab) and SonarCloud
Latest source code from main branch
original
An exact export from the classic pipeline
Code coverage is displayed both in Azure Devops (Extensions-tab) and SonarCloud
Not pulled latest source code from main branch
replica
An exact export from the classic pipeline, identical to original pipeline
Code coverage lacks from both in Azure Devops (Extensions-tab) and SonarCloud
Latest source code from main branch
To be clear: The tasks in the pipeline doesn’t fail, but code coverage isn’t displayed as I expect. I expect the measure code coverage to be here and in SonarCloud, but it isn’t for the replica pipeline. I’m very confused, because the replica and original pipeline have identical setup, differing only in source code, while the classic and replica pipeline have identical source code and should have identical setup too. But yet the replica pipeline fails, and I cannot figure out why.
I might add that I’ve noticed that the published artifact varies from the cases I’ve mentioned:
Pipeline name
Contents of drop-folder (for published build artifacts)
Size of content
original
.sqAnalysis-folder, containing SonarQubeBuildSummary.md and zip of project
.sqAnalysis (3KB), zip-file 26 MB
replica
.sqAnalysis-folder, containing SonarQubeBuildSummary.md and zip of project
.sqAnalysis (2KB), zip-file 24 MB
classic
Zip-file of project
23 MB
Moreover, there seems to be some caching issues in SonarCloud, as the numbers in overview and branch summary differs. Is it intentionally? Or are there any ways to bust the cache?
From overview:
This gave an error message stating the runsettings-file couldn’t be found. So I temporarily removed the reference to the runsettings-file, and then I got code coverage stats - not for the new code, but for the main branch. But I want stats for new code only. Do you have any suggestions on how to solve this?
This doesn’t surprise me, as I’m pretty sure the SCM checkout is tucked away somewhere else when configuring a classic pipeline, so the config might vary when you migrate to YAML.
I’m however really surprised to hear you got an error related to .runsettings just because you changed the clone… Does that .runsettings file only exist on your new branch, and not on the main branch?
DId you add these at the same time? I think you only need to add the latter.
@Colin No, the runsettings-file has been there for a few months, so it should be available in both branches. And I previously only had
- checkout: self
fetchDepth: 0
but as mentioned I tried adding the other checkout to resolve the SCM-error. I also tried changing the order, but the error was still there. Maybe there is some other way to resolve the SCM-error, and maybe checkout is not it. At least if it is an alias to git checkout, it would mean that you checked out the branch, i.e. in effect switched branches, but what we really want is a fetch or pull from the main branch. I could check if the pipeline runs on the main branch, but it should run on branches that stems from the main branch, so I’m not sure if that would actually resolve anything.
Also, regarding the SCM error … Our main branch is called main, but the error message mentions master, could that mean something?
You might be onto something here. Could you try setting sonar.branch.target=main in your build pipeline for a test branch to see if that changes things?
I have a feeling something might need improvement in our Azure DevOps extension to, at least, set sonar.branch.target to the default branch (today it’s not doing that, and defaulting to master).
