Template for a good new topic, formatted with Markdown:
ALM used (GitHub, Bitbucket Cloud, Azure DevOps)
Bitbucket
CI system used (Bitbucket Cloud, Azure DevOps, Travis CI, Circle CI
Argo
Scanner command used when applicable (private details masked)
Languages of the repository
Python
Only if the SonarCloud project is public, the URL
And if you need help with pull request decoration, then the URL to the PR too
Error observed (wrap logs/code around with triple quotes ``` for proper formatting)
In the project settings I see some options for static code coverage. Those test the code for the current state.
What I need is a setting to compare the code coverage of the current pull request to that of the main branch. Code coverage should not go down. Which of the code coverage settings could help me with that?
Except for observing the Estimated after Merge measure on the PR dashboard, it’s not possible to restrict the merge of a PR based on how the coverage compares to the target branch.
That being said, the idea of “ratcheting” Quality Gates has come up time and time again (for measures that must stay the same or improve, and should not decrease). I think I even saw an internal request for this a few days ago. I’m happy to note your feedback.
so it is only possible to define a quality gate for the test coverage “after Merge” but not in comparison to “before Merge”?
Isn’t this what the other metrics already support? But not custom metrics like test coverage?
If this is in fact currently not implemented, is there any ETA on this feature? That was the main point why we decided for SonarQube to ensure that the test coverage is not getting worse. I’m surprised that apparently this is not supported right now.
On Pull Requests, it is possible (and is the case by default) that Coverage on New Code must meet certain criteria (in the default Sonar Way Quality Gate, the coverage threshold is set to 80%).
This should, in almost all cases, ensure that coverage on new code is always meeting a certain threshold. As long as that threshold is equal to or higher than the coverage on your overall codebase, you can be pretty sure coverage won’t decrease when the branch is merged.
However, this does not prevent a scenario where, for example, there were removed/failing tests (which influence the coverage figure as well). This could impact the coverage post-merge, but not be reported on the Pull Request if it doesn’t impact New Code.
Just to confirm that I’m getting this right, the Coverage on New Code is actually not the total test coverage over the new state of the whole code but actually just the test coverage of the touched / changed code lines?
So for example, if I only change a single line of code, it would check if this single line of code is covered with tests. If yes, the reported Coverage on New Code would be 100% and if no it would be 0%?
If yes, I think this would also fulfill our needs.
