How to set quality gate on PR based on estimated after merge

I know we can have quality gate based on coverage but that’s a hard coded value. What I want to enforce is it that the total code coverage should not decrease.

For example I set the QG on coverage to be > 50% on new code.
PR A increased the total coverage to 55%.
Now PR B should be checked to have coverage > 55%, meaning it should not decrease the coverage from it’s current value.

I hope I’m clear enough. I’m using SonarQube version 8.1.

Hi,

This is a good one! Unfortunately, I don’t have a good answer for you. Or at least, I don’t have an answer you’re going to like.

First, Quality Gate conditions are set on metrics/measures but there’s not a metric for “estimated coverage after merge”; it’s just math. I.e. we don’t calculate and store that value. We only calculate and display it.

But even if it were a metric, I think it would count as an “overall” metric rather than an “on New Code” metric, and thus it wouldn’t be eligible for use in a PR Quality Gate.

Nonetheless, I’ve referred this internally. It’s possible that something might happen here eventually. But for the foreseeable future, the answer is “I’m sorry, no.”.

 
Ann

2 Likes

In the case described above, the quality gate has lower standards (50%) than the actual code coverage (55%) so it introduces a risks the over coverage reduces, in which case such feature could be welcome.

But if a quality gate on new code is always with higher standards than actual overall coverage, is it safe to say the overall coverage can only increase? In such case the requested behaviour would be unecessary, am I correct?

Gabriel.

Hi,

Emphasis added

Yes, I think it is.

For background, that’s exactly what we’ve experienced internally at SonarSource. We set a QG condition on Coverage on New Code, and as a side-effect(? result?) gradually saw our overall coverage on SonarQube itself increase.

 
HTH,
Ann

You could just remove tests and Sonar would allow to merge. Also changes on tests can decrease coverage for existing code and Sonar will only check if new code is covered.
We see this happening from time to time and would like to also enforce a condition on the expected total coverage.