Sonar Cloud analysis on Azure DevOps pipeline

We’ve been successfully using the three tasks (SonarCloudPrepare, SonarCloudAnalyze and SonarCloudPublishQualityGate) you provide for quality assessment (with dotnet test to generate test coverage reports and running our unit tests) for a few months now. Last week code coverage started failing, both for our onprem and cloud solutions. I’m transforming our Azure DevOps build pipelines from classic to yaml, and today I discovered new versions of the Sonar tasks (which I can’t recall seeing before) and updated them in the classic pipelines. Updating those tasks partly resolved my issue.

The classic pipelines now have code coverage, both in SonarCloud and in the extension-tab on the pipeline. So I tried exporting the yaml from the classic pipeline, and wouldn’t you know? It worked! That is, until I pulled the latest commits from our main branch (which the classic pipeline handles withour errors). Then the Sonar analysis report on the extension tab and SonarCloud lacked information about test coverage.

I find it very odd, as the pipeline steps in both cases should be equal, and I can’t see that the updated code should hinder code coverage in Sonar (the updated code is just nuget package updates, which, as I’ve already’ve mentioned, hasn’t prevented code coverage for the classic pipeline). The other tabs in the pipeline shows code coverage and test results correctly, so I’m trying to find out why quality gate behaves this way. Any ideas? I see that the information in the extension tab stems from the build’s artifactstaging-directory in a markdown file, but I can’t find how that file or other data passed to sonarcloud’s quality gate.

  • ALM used (Azure DevOps)
  • CI system used (Azure DevOps
  • Scanner command used when applicable (private details masked): MSBuild
  • Languages of the repository: C#

Hey there!

It looks like you were able to solve the issue based on your post here:

And that’s exactly what I was going to recommend – upgrade to v2 of the tasks.

I’m not quite sure what you mean by this. Can you come back to the original thread with logs from your failing pipelines?

Can do.

Was redirected from the other thread, so reposting here:

To give further info:
Until a few weeks ago, we had around 10 pipelines for different system. These pipelines use the classic pipeline from Azure DevOps. But we have a goal to replace these classic pipelines with yaml pipelines, meaning the pipelines should produce the same results. However, once I fixed the code coverage issue for the classic pipelines, I haven’t fully been able to do so with the yaml pipelines (yet). This is what I’ve observed so far:

Pipeline name Description Result Source code status
classic Original classic pipeline Code coverage is displayed both in Azure Devops (Extensions-tab) and SonarCloud Latest source code from main branch
original An exact export from the classic pipeline Code coverage is displayed both in Azure Devops (Extensions-tab) and SonarCloud Not pulled latest source code from main branch
replica An exact export from the classic pipeline, identical to original pipeline Code coverage lacks from both in Azure Devops (Extensions-tab) and SonarCloud Latest source code from main branch

To be clear: The tasks in the pipeline doesn’t fail, but code coverage isn’t displayed as I expect. I expect the measure code coverage to be here and in SonarCloud, but it isn’t for the replica pipeline. I’m very confused, because the replica and original pipeline have identical setup, differing only in source code, while the classic and replica pipeline have identical source code and should have identical setup too. But yet the replica pipeline fails, and I cannot figure out why.
image

I might add that I’ve noticed that the published artifact varies from the cases I’ve mentioned:

Pipeline name Contents of drop-folder (for published build artifacts) Size of content
original .sqAnalysis-folder, containing SonarQubeBuildSummary.md and zip of project .sqAnalysis (3KB), zip-file 26 MB
replica .sqAnalysis-folder, containing SonarQubeBuildSummary.md and zip of project .sqAnalysis (2KB), zip-file 24 MB
classic Zip-file of project 23 MB

Moreover, there seems to be some caching issues in SonarCloud, as the numbers in overview and branch summary differs. Is it intentionally? Or are there any ways to bust the cache?
From overview:
image

Main branch summary:

@Beate Probably the best thing to do is to run a diff between the logs of your classic and YAML pipelines, and look for obvious differences.

A tool like this is often useful (best if you can get rid of the timestamps in the logs, but even then, still useful) https://www.diffchecker.com/

You’ll want to look at all the tasks – because the difference could be in test execution, or generating the code coverage report, or reading it.

@Colin I did a diffcheck and saw that in the pipeline where the code coverage didn’t work, this line was present:

Could not find ref: master in refs/heads, refs/remotes/upstream or refs/remotes/origin

Which in a way made sense. The classic pipeline is run on main, and the yaml pipeline without code coverage is run on another branch.

So i tried adding

 - checkout: <path to repo>
    fetchDepth: 0
 - checkout: self
    fetchDepth: 0

This gave an error message stating the runsettings-file couldn’t be found. So I temporarily removed the reference to the runsettings-file, and then I got code coverage stats - not for the new code, but for the main branch. But I want stats for new code only. Do you have any suggestions on how to solve this?

Hey @Beate

This doesn’t surprise me, as I’m pretty sure the SCM checkout is tucked away somewhere else when configuring a classic pipeline, so the config might vary when you migrate to YAML.

I’m however really surprised to hear you got an error related to .runsettings just because you changed the clone… Does that .runsettings file only exist on your new branch, and not on the main branch?

DId you add these at the same time? I think you only need to add the latter.

 - checkout: self
    fetchDepth: 0

@Colin No, the runsettings-file has been there for a few months, so it should be available in both branches. And I previously only had

- checkout: self
    fetchDepth: 0

but as mentioned I tried adding the other checkout to resolve the SCM-error. I also tried changing the order, but the error was still there. Maybe there is some other way to resolve the SCM-error, and maybe checkout is not it. At least if it is an alias to git checkout, it would mean that you checked out the branch, i.e. in effect switched branches, but what we really want is a fetch or pull from the main branch. I could check if the pipeline runs on the main branch, but it should run on branches that stems from the main branch, so I’m not sure if that would actually resolve anything.

Also, regarding the SCM error … Our main branch is called main, but the error message mentions master, could that mean something?