Must-share information (formatted with Markdown):
- which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension):
- SonarQube Enterprise Edition Version 8.0 (build 29455)
- what are you trying to achieve
- Consistent Quality Gate behaviour
- what have you tried so far to achieve this
Is more detailed information available on the behaviour of Quality Gates for Short Lived vs Long Lived vs Master Branch? Or is someone able to explain the diffent behaviour we are seeing.
I’ve read the information available here:
https://docs.sonarqube.org/latest/branches/short-lived-branches/
However the behaviour is not as I would expect. We currently have the following Conditions setup for our quality gates
However the metrics which are returned for a Short Lived Branch, Long Lived branch and a Master branch is not what we expect. Below are the various metrics we are getting back for the various types of branches.
Short lived branch
https://sonarqube/api/qualitygates/project_status?analysisId=AW7FMdHrrVkB8MhlBehW
{“projectStatus”:{“status”:“OK”,“conditions”:[{“status”:“OK”,“metricKey”:“new_reliability_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_security_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_maintainability_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_duplicated_lines_density”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“3”,“actualValue”:“0.0”}],“periods”:,“ignoredConditions”:false}}
Long lived branch
https://sonarqube/api/qualitygates/project_status?analysisId=AW7JvzmbrVkB8MhlBknC
{“projectStatus”:{“status”:“OK”,“conditions”:[{“status”:“OK”,“metricKey”:“new_reliability_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_security_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_maintainability_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_duplicated_lines_density”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“3”,“actualValue”:“0.0”},{“status”:“OK”,“metricKey”:“test_errors”,“comparator”:“GT”,“errorThreshold”:“0”,“actualValue”:“0”},{“status”:“OK”,“metricKey”:“test_failures”,“comparator”:“GT”,“errorThreshold”:“0”,“actualValue”:“0”}],“periods”:[{“index”:1,“mode”:“PREVIOUS_VERSION”,“date”:“2019-12-03T09:45:24+1100”}],“ignoredConditions”:false}}
Master
https://sonarqube/api/qualitygates/project_status?analysisId=AW7EHuxtrVkB8MhlBGNN
{“projectStatus”:{“status”:“OK”,“conditions”:[{“status”:“OK”,“metricKey”:“new_reliability_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_security_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“new_maintainability_rating”,“comparator”:“GT”,“periodIndex”:1,“errorThreshold”:“1”,“actualValue”:“1”},{“status”:“OK”,“metricKey”:“test_errors”,“comparator”:“GT”,“errorThreshold”:“0”,“actualValue”:“0”},{“status”:“OK”,“metricKey”:“test_failures”,“comparator”:“GT”,“errorThreshold”:“0”,“actualValue”:“0”}],“periods”:[{“index”:1,“mode”:“PREVIOUS_VERSION”,“date”:“2019-11-30T16:33:39+1100”}],“ignoredConditions”:false}}
Master (The first time)
https://sonarqube/api/qualitygates/project_status?analysisId=AW66zQsZrVkB8Mhl_7_8
{“projectStatus”:{“status”:“OK”,“conditions”:[{“status”:“OK”,“metricKey”:“test_errors”,“comparator”:“GT”,“errorThreshold”:“0”,“actualValue”:“0”},{“status”:“OK”,“metricKey”:“test_failures”,“comparator”:“GT”,“errorThreshold”:“0”,“actualValue”:“0”}],“periods”:,“ignoredConditions”:false}}
This table summarises the metrics we are receiving for our Quality gate setting, vs what we would have expected to get.
Conditions on New Code Short Lived Long Lived Branch Master Branch Master Branch First time Expect for short lived Expect for Long lived
Coverage new_coverage y y
Duplicated Lines % new_duplicated_lines_density y y y y
Maintainability Rating new_maintainability_rating y y y y y
Reliability Rating new_reliability_rating y y y y y
Security Rating new_security_rating y y y y y
Condtions on Overall Code
Unit Test Errors test_errors y y y y
Unit Test Failures test_failures y y y y
We never get a value for new_coverage, in the Master branch we also never got a new_duplicated_lines_density. Oddly on Master the first time an analysis was performed we only got back values for test_errors and test_failures