Quality Gate on MAIN BRANCH failed because of 3 new lines of code without coverage

SonarQube ID information
Server ID: 4E0B12C0-AWx3FFVJ-rodZ4C38Nhp
Version: 8.4.2.36762
Date: 2021-02-03

The Quality Gate passed on the Pull Request:

But failed once merge in the main branch, because of the coverage on 3 new lines of code not ignored:

Hello @fmunteanu ,

Welcome to the SonarSource community. :wave: . I hope you’ll enjoy it

The behaviour that you describe is expected, although debatable.
As you may have notice we have a special behaviour on PRs that we don’t consider the code coverage criteria for the QG it there are less than 20 lines added in the PR. The reason is that if a developer does a hotfix adding only a few lines on some code that has 0 code coverage it’s a bit unfair (and generates a lot of work) to ask the developer to cover his few lines (he will have to put in place the whole test framework…)
This criteria does not apply on the target branch after merge. So the quality gate fails if you have even a single line to cover, and that it is not covered.

So requiring coverage even for a few lines on PR makes sense in some cases and may be too tough in others. Difficult to meet the need of all cases. Right now the minimum lines to consider coverage for QG on PR is 20 and not configurable. In the future we consider making this parameter configurable so that you can choose between a strict policy on PR (even if the PR introduces a few lines, coverage should be good), or a more relaxed one (below a certain number of lines, coverage is not mandatory on PRs)

In the mean time you can only rely on you developers discipline to add coverage on small PRs even if the quality gate does not enforce it

Olivier

Hello Oliver, thank you for the quick and details answer.

Unfortunately for us it can be a big issue, since the Developer in charge of the Merge Review sees the Quality Gate and the CI pipeline “GREEN” on the feature branch, but once they merge into main branch, the pipeline becomes “RED”.

We are forced to apply workarounds like setting a temporary QG with coverage 0% just to unblock the main branch, and then set back the preview QG.

In our case at least it will be better to block the feature/bugfix branch prior to merge in main branch.
So that other merge requests can be still merged in the mean time.

Is there any other solution to this problem?

Thanks,
Francisc

Hello Francisc,

I understand. I don’t have any good solution to propose.
I can suggest a bad one in case it works for you: You add the Quality Gate Criteria: uncovered lines on new code > 0. This will fail the QG as soon as one line is not covered… so this will force 100% coverage on the PR. I am saying it’s a bad option because:

  • It requires 100% coverage on the PR which is often overkill
  • It will require 100% on the target branch after merge too which is even more overkill

I think the best is to have a temporary process where the reviewed checks the coverage (not only the QG)… until we implement https://jira.sonarsource.com/browse/SONAR-10485.
I cannot commit but this may come relatively soon.

Thank you Oliver,
is there any expected completion date for this feature?
I see was opened several years ago and it’s still unassigned :slightly_frowning_face:?

Francisc

Hello Francisc,

As I said on my previous post,

Olivier

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.