Not all quality gate metrics are given via SonarQube API

[Eva]

Hi,

we had configured a quality gate with 8 conditions in total (4 conditions for new code & 4 conditions for overall code).

But if we want to include the information in GitLab via SonarQube API call xxx/api/qualitygates/project_status?projectKey it only shows 7 metrics (I guess ‘new_security_hotspots_reviewed’ is missing.)

Is there a bug?

We are using Enterprise Edition v10.6 (92116).

Thanks for any hints.

Greetings,
Eva

[Colin]

Hey @Eva

What’s probably going on is that if you have 0 new security hotspots… it’s not possible to calculate what percentage of new security hotspots are reviewed, because you’d be dividing by zero.

I don’t disagree that it’s confusing, and I wonder what would make sense for you to appear in the API response for measures that aren’t calculated due to a situation like above.

[Eva]

The metric ‘new_security_hotspots_reviewed’ is given in percentage of reviewed security hotspots on new code. I assumed that 100.0 % is the success rate if …
a) no security hotspot has to be reviewed
b) all pop-up security hotspots were reviewed

Or?

My assumption is:
If I configure 8 conditions, I want to get 8 measurements for those metrics (could be also 0 ) including the status OK or ERROR for the metric.

[vivek.reghunath]

Thanks Eva for bringing this to our notice. Will look into it.

[Wouter_Admiraal]

Hi @Eva ,

Sorry to come back to this a bit late.

Did you add that new_security_hotspots_reviewed condition after the last project analysis?

The way api/qualitygates/project_status works is that it returns a “snapshot” of the quality gate evaluation when the project was last analyzed (OK, it’s actually slightly more complicated, but for the sake of this illustration let’s assume that’s how it works ). If you update your Quality Gate definition (e.g., add or remove conditions), this won’t be reflected in api/qualitygates/project_status unless the project is analyzed again.

On my side, when I:

1. Create a Quality Gate just like the one you show in your screenshot, but omit the new_security_hotspots_reviewed condition.
2. Assign it to a project and analyze it.
3. Then update the Quality Gate and add the missing new_security_hotspots_reviewed condition, but don’t reanalyze the project.
4. If I fetch the JSON payload again, I see the new condition is missing (as in your example).

   {
       "projectStatus": {
           "status": "ERROR",
           "conditions": [
               {
                   "status": "OK",
                   "metricKey": "new_software_quality_maintainability_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "1"
               },
               {
                   "status": "ERROR",
                   "metricKey": "new_software_quality_reliability_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "3"
               },
               {
                   "status": "OK",
                   "metricKey": "new_software_quality_security_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "1"
               },
               {
                   "status": "ERROR",
                   "metricKey": "security_hotspots_reviewed",
                   "comparator": "LT",
                   "errorThreshold": "100",
                   "actualValue": "0.0"
               },
               {
                   "status": "OK",
                   "metricKey": "software_quality_maintainability_rating",
                   "comparator": "GT",
                   "errorThreshold": "3",
                   "actualValue": "1"
               },
               {
                   "status": "OK",
                   "metricKey": "software_quality_reliability_rating",
                   "comparator": "GT",
                   "errorThreshold": "3",
                   "actualValue": "3"
               },
               {
                   "status": "OK",
                   "metricKey": "software_quality_security_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "1"
               }
           ],
           "ignoredConditions": false,
           "period": {
               "mode": "PREVIOUS_VERSION",
               "date": "2023-09-26T15:36:04+0200"
           },
           "caycStatus": "non-compliant"
       }
   }
   

5. I reanalyze the project and fetch the JSON payload again, and now the missing condition is there.

   {
       "projectStatus": {
           "status": "ERROR",
           "conditions": [
               {
                   "status": "ERROR",
                   "metricKey": "new_security_hotspots_reviewed",
                   "comparator": "LT",
                   "errorThreshold": "100",
                   "actualValue": "0.0"
               },
               {
                   "status": "OK",
                   "metricKey": "new_software_quality_maintainability_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "1"
               },
               {
                   "status": "ERROR",
                   "metricKey": "new_software_quality_reliability_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "3"
               },
               {
                   "status": "OK",
                   "metricKey": "new_software_quality_security_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "1"
               },
               {
                   "status": "ERROR",
                   "metricKey": "security_hotspots_reviewed",
                   "comparator": "LT",
                   "errorThreshold": "100",
                   "actualValue": "0.0"
               },
               {
                   "status": "OK",
                   "metricKey": "software_quality_maintainability_rating",
                   "comparator": "GT",
                   "errorThreshold": "3",
                   "actualValue": "1"
               },
               {
                   "status": "OK",
                   "metricKey": "software_quality_reliability_rating",
                   "comparator": "GT",
                   "errorThreshold": "3",
                   "actualValue": "3"
               },
               {
                   "status": "OK",
                   "metricKey": "software_quality_security_rating",
                   "comparator": "GT",
                   "errorThreshold": "1",
                   "actualValue": "1"
               }
           ],
           "ignoredConditions": false,
           "period": {
               "mode": "PREVIOUS_VERSION",
               "date": "2023-09-26T15:36:04+0200"
           },
           "caycStatus": "non-compliant"
       }
   }
   

Did the missing condition appear once your project was reanalyzed?