SonarQubePublish@7 - Set task complete result based on Quality Gate result

SonarQube Cloud
1

I am using the SonarQubePublish@7 - Publish Quality Gate Result v7 task with Azure DevOps Services.

I have noticed behavior related to the SonarQube Quality Gate that results that is not ideal when using this task in Azure DevOps.

Regardless if the publish task sends results where the Quality Gate passes or fails, the task always exits in Success.

I was hoping you would consider adding an optional parameter to the task, something along the lines of:

parameters:
- name: qualityGateFailedTaskResult
  type: string
  values:
    - Succeeded
    - SucceededWithIssues
    - Failed
  default: 'Succeeded'

Then inside your task, when it detects that the quality gate fails, as shown in the log below:

##[section]Starting: Publish Quality Gate Result
==============================================================================
Task         : Publish Quality Gate Result
Description  : Publish SonarQube Server's Quality Gate result on the Azure DevOps build result, to be used after the actual analysis.
Version      : 7.3.0
Author       : sonarsource
Help         : [More Information](https://docs.sonarsource.com/sonarqube/latest/analyzing-source-code/scanners/sonarqube-extension-for-azure-devops/)
==============================================================================
[INFO]  SonarQube Server: Server version: 10.3.0.82913
[INFO]  SonarQube Server: Task AZ***-W****z_6*****u completed
[INFO]  SonarQube Server: Overall Quality Gate status: failed
##[section]Finishing: Post-Build: Publish SonarQube Quality Gate Result

When the code is writing ‘[INFO] SonarQube Server: Overall Quality Gate status: failed’, you could use the value that tracks overall quality gate status to control the task complete result with ##vso console command.

This could be done at the end of the task execution, based on that value and the new parameter, you could write to the console output this command (I’m using pwsh and standard YML syntax in my example):


$overallQualityGateStatus = 'failed' # set by your existing code
$taskCompleteResult = '${{ parameters.qualityGateFailedTaskResult }}'

if ($overallQualityGateStatus -eq 'failed' -and $taskCompleteResult -ne 'Succeeded') {
    Write-Host "##vso[task.complete result=$taskCompleteResult;]Overall Quality Gate status failed"
}

Here is another form of the example which would probably be more inline with your node based task codebase (I am basing this off memory of writing my own node tasks, I did not test it but it should give the general idea):

const tl = require('azure-pipelines-task-lib/task');

const overallQualityGateStatus = tl.getVariable('overallQualityGateStatus');
const taskCompleteResult = tl.getInput('qualityGateFailedTaskResult', true);

if (overallQualityGateStatus === 'failed' && taskCompleteResult !== 'Succeeded') {
    tl.setResult(tl.TaskResult[taskCompleteResult], 'Overall Quality Gate status failed');
} else {
    tl.setResult(tl.TaskResult.Succeeded, 'Overall Quality Gate status passed');
}

If you were to add functionality like this, it would give DevOps teams a lot more flexibility into how Quality Gate failures effect our project build results.

With this in place, on a legacy project, we can use the default value (which would be the current behavior for backwards compatibility). In this case, even if the Quality Gate fails, the task would complete with Succeeded and the build process would continue allowing the project to be deployed in a CI/CD workflow for example.

However, if we wanted to give more visibility to the fact the Quality Gates are failing, but not go so far as failing the builds, we could use the option ‘SucceededWithIssues’. With this option, our builds would complete, the CI/CD workflow would proceed, but in our build history we would see the build had issues. This make it more obvious to the team that further investigation into the build issues is needed. This is the middle ground option: shows us we have quality issues but does not fail the build. Useful for legacy code we are working to improve for example or for teams that do not have a zero tolerance on quality gate failures.

The most strict option, ‘Failed’, would allow us to have Quality Gate failures cause a build failure, preventing the code from being deployed by our CI/CD workflow. This would be useful in modern projects that already have consistent clean scans with SonarQube, so that when a quality gate does fail, we can block the code from moving forward.

I have implemented similar functionality in our build pipelines for SCA scans as well as unit test results, so they at least flag as having ‘SucceededWithIssues’ but with a strict mode option to cause them to ‘Fail’.

I had to make those tasks from scratch, but as you provide these tasks already, it would be great if this functionality could be added by your team to the task.

I am sure others like myself would find great use in being able to control build results based on your quality gate data, so I hope you consider adding this feature to your tasks. I am pretty sure it would be a very minor change for you to implement.