Azure DevOps integration broken

SonarQube
,
1
  • SonarQube server version - 7.2, SonarQube VSTS extension version 4.7.2
    We have Azure DevOps pipeline which has code analysis step which integrates SonarQube VSTS extension to analyze .NET C# project by calling the following tasks
  1. SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4
    with the following extra properties
    sonar.cs.vscoveragexml.reportsPaths=*.coveragexml
    sonar.cs.vstest.reportsPaths=    *.trx
  2. MSBuild@1
  3. VSTest@2 with codeCoverageEnabled: true
  4. SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4
  5. SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4

All the above tasks complete successfully and the build completes successfully. But the code coverage information and the static analysis results are not getting published on the SonarQube server.

I see that if I run the analysis on a new branch, that new branch is created on the SonarQube server but the report contains no violations though we intentionally introduced unused functions and variables which should have been reported.

Anyone else facing similar issues with Azure DevOps integration?

1 Like
2

Update on this as the issue is still not resolved but we are unable to find out the root cause. There seems to be more than one issue but yet to figure out where to fix the issue (Azure DevOps side or SonarQube side).

The number of unit test shows 0 but the build contains more than 1500 unit test cases. This seems to be the main reason why the code coverage is not showing up. Any idea what could be the reason the count shows 0 when the coverage file downloaded from the same build opens properly in Visual Studio? If we are able to resolve this, then the code coverage issue should get resolved automatically.

Regarding pull request decoration, I could see the following warning message in the SonarQube Command Engine logs

2019.09.09 05:28:11 WARN ce[AW0UflhaIAgssz1eoTKa][c.s.B.C.A.C] Failed to decorate VSTS Pull Request: API resource location 225f7195-f9c7-4d14-ab28-a83f7ff77e1f is not registered on https://AzureDevOpsServerUrl. HTTP 401 Unauthorized

If all the SonarQube tasks are completing successfully and the branch is getting created successfully on the SonarQube server, I am not able to understand why the API resource location not registered error is coming up with 401 Unauthorized. We have even regenerated both Azure DevOps token (with full access) and also SonarQube token and updated in respective settings and still the above error is not getting resolved. Due to this, we are not able to see the Pull Request Decoration after a successful CI build.

So it looks like there are two different problems that could be contributing to the problems we are seeing.

Please provide any helpful pointers to resolve these issues.

3

Hi,

What is your Azure DevOps version ? Is that an old VSTS or a 2019 AzDO ?

Thanks.

Mickaƫl

4

It is on Microsoft hosted cloud https://dev.azure.com. Please let me know if this information is sufficient or if there is a way to find out the version of this cloud instance.

5

Hi, Thanks.

We created a ticket internally as it appears that multiple people face this issue.

Investigating on it, we will let you know.

Mickaƫl

6

For troubleshooting and trying to reproduce this issue on my side, can you please provide

  • the setup you have on the Status Policy for SonarCloud/ quality gate

image
image.png475Ɨ854 23.5 KB

  • Do you have multiple status policies within the branch ?
  • What is the scope of the PAT that is recorded on SonarCloud ?

Thanks !

7

Thanks for the update.

We have not set any status policy for ā€œSonarQube/QualityGateā€ but there is a second policy related to CodeCoverage which is set to Optional for all branches

Regarding the CodeCoverage status policy we are having a different issue which I do not want to ask here, but because it is related to the policy, I wanted to provide that info as well just in case it is related/helpful. Few weeks ago, we saw that the Code Coverage policy got added as ā€œRequiredā€ for all pull requests and I am not able to find out how and who set that code coverage policy to required and so I am not sure how to remove that. As a workaround, we have set the Branch policy to ā€œOptionalā€ to override that global policy.

With respect to Scope of the PAT token, we have tried with just ā€œCode Read/Writeā€ and when that did not help we even tried ā€œFull Accessā€ and still the pull request decoration is not successful.

8

By the way, which issue is observed by many others? Is it related to code coverage showing up as 0 with 0 unit test cases? Or the Pull Request Decoration failure with 401? Or both?

9

Also just to be sure, we are using SonarQube, not SonarCloud.

10

The code coverage status is a newest feature added in sprint 155 (https://docs.microsoft.com/en-us/azure/devops/release-notes/2019/sprint-155-update#code-coverage-metrics-and-branch-policy-for-pull-requests) and itā€™s showing up by default in all PRs.

The issue faced by the other is the unauthorized PR status, not the coverage. By the way, if you are using SonarQube, how is made the communication between SQ and Azure ? For information, even if itā€™s working, we have a dedicated extension with SonarCloud and are not supporting these kind of scenario.

For coverage issues, number of unit tests and coverage are decorelated since one is provided by the .trx and the other one by *.coveragexml files.
Are you seeing something in the Run Code Analysis log like :

Parsing the Visual Studio coverage XML report \..\.coveragexml
INFO: Adding this code coverage report to the cache for later reuse:

You can as well try with debug mode and see if your projects are not categorised as ā€œTest projectsā€ (the one that are not Unit Tests nor Integration Tests obvisouly)

Thanks.

Mickaƫl

11

Thanks for clarifying (and the link) about the Code Coverage metrics that was added automatically.

The connection between SonarQube and Azure is made using ServiceConnection. We generate a token from SonarQube server and provide that token in the ServiceConnection.

Regarding SonarCloud extension, I am not clear about what you had suggested. Is the suggestion to use the SonarCloud extension instead of SonarQube extension even though the server is running SonarQube?

Regarding the log file for Run Code Analysis, I am not seeing the line about Parsing the coveragexml statement but something similar was found

2019-09-09T06:13:28.5811465Z 06:13:28.569 Attempting to locate the CodeCoverage.exe toolā€¦
2019-09-09T06:13:28.5852903Z 06:13:28.569 Attempting to locate the CodeCoverage.exe tool using setup configurationā€¦
2019-09-09T06:13:28.6031433Z 06:13:28.6 Code coverage command line tool: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
2019-09-09T06:13:28.6062909Z 06:13:28.6 Property ā€˜sonar.cs.vstest.reportsPathsā€™ provided, skipping the search for TRX files in default foldersā€¦
2019-09-09T06:13:29.3280893Z 06:13:29.32 Did not find any binary coverage files in the expected location.
2019-09-09T06:13:29.3292934Z 06:13:29.32 Falling back on locating coverage files in the agent temp directory.
2019-09-09T06:13:29.3301009Z 06:13:29.32 Searching for coverage files in D:\a_temp
2019-09-09T06:13:29.3331738Z 06:13:29.32 All matching files: count=2
2019-09-09T06:13:29.3331945Z 06:13:29.32 D:\a_temp\TestResults\93a72db4-2190-44af-978b-b0767817d6f3\VssAdministrator_fv-az639_2019-09-09.06_11_58.coverage
2019-09-09T06:13:29.3332633Z 06:13:29.32 D:\a_temp\TestResults\VssAdministrator_fv-az639_2019-09-09_06_12_10\In\fv-az639\VssAdministrator_fv-az639_2019-09-09.06_11_58.coverage
2019-09-09T06:13:29.3337265Z 06:13:29.32 Unique coverage files: count=1
2019-09-09T06:13:29.3337457Z 06:13:29.32 D:\a_temp\TestResults\93a72db4-2190-44af-978b-b0767817d6f3\VssAdministrator_fv-az639_2019-09-09.06_11_58.coverage
2019-09-09T06:13:29.5013235Z 06:13:29.492 Executing file C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
2019-09-09T06:13:29.5013835Z Args: analyze /output:D:\a_temp\TestResults\93a72db4-2190-44af-978b-b0767817d6f3\VssAdministrator_fv-az639_2019-09-09.06_11_58.coveragexml D:\a_temp\TestResults\93a72db4-2190-44af-978b-b0767817d6f3\VssAdministrator_fv-az639_2019-09-09.06_11_58.coverage
2019-09-09T06:13:29.5014677Z Working directory: D:\a_temp\TestResults\93a72db4-2190-44af-978b-b0767817d6f3
2019-09-09T06:13:29.5014772Z Timeout (ms):60000
2019-09-09T06:13:29.5014838Z Process id: 5276
2019-09-09T06:13:30.5825161Z 06:13:30.557 Process returned exit code 0

12

As long as you let the Scanner for MSBuild converting for you the binary files, you donā€™t need this setting. (it appears to be the case given the last log you gave)

So try to remove the setting and see if the coveragexml files gets uploaded to SonarQube (you can also add system.debug=true to your build variable, and send the output (here or by message if you have sensitive information inside)

Thanks.

13

Even after removing sonar.cs.vscoveragexml.reportsPaths parameter, we are not getting the code coverage data

14

But you had it before ?

Iā€™m sorry, i think we are melting topics here, and iā€™m a bit confused.

Can you re-clarify what is your exact issue ?

Thanks.

15

We are having multiple issues

First issue is with respect to code coverage. We had the sonar.cs.vscoveragexml.reportsPaths parameter always from the beginning. We were getting code coverage data till a month ago and now the code coverage data shows all lines as ā€œuncoveredā€.

The second problem is with respect to pull request decoration which is not pushing any data back to pull requests due to a warning that the last analysis pull request decoration failed due to 401 Unauthorized. We are unable to resolve this in spite of replacing tokens with fresh value, full scope, future date for expiration.

17

We were able to resolve this issue by prefixing the $(Agent.TempDirectory) to the path of coveragexml and the .trx file as shown below. It looks like the path of these files got moved to Agentā€™s temp folder due to which the files were not getting transferred which was causing coverage as well as number of unit tests to be 0

extraProperties: |
              sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)\TestResults\**\*.coveragexml
              sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)\TestResults\**\*.trx
1 Like
18

Hi @srikcgaa3 ,

Yes, indeed, the TestResults folder was moved by Microsoft when the VSTest was updated to version 2.

I didnā€™t see the initial issue, but, as per the doc, the single star wildcard on all the properties targets only files, and not directories. Thus, it as per your situation, it will look only on the root directory, and not recursively. You were totally right to add double stars (= find in any directories) and Agent.TempDirectory variable.

Thank you.

Mickaƫl

19

@srikcgaa3 did you manage to get your second issue fixed?
(The 401 when decorating the PRā€™s in Azure Devops)

We are also seeing this problem, which seem to have started after switching from one SonarQube server to another.

20

Thank you for this, I was having exactly the same issue!

21

Hi @SierraNL, A bit late reply, sorry about that. In our case, we had to replace the PAT token from a different account that had organization administrator privilege and that solved. If your scenario was also fixed, could you please post your solution as well?