I had problems with our build system today, so I only just now tried testing a pull request that reduces the percentage by adding a bunch of untested code, instead of adding an @Ignore to tests. I assumed that this would work better. The result, however, is perplexing. I added a single source file with a single method, which contains a large number of println statements.
Here are some pictures of what the result was:
Am I missing something obvious here?
FYI @David_Karr I’ve split this to a new topic because I believe it’s a fundamentally different question.
Now, the question is are you passing in a coverage report and are any lines in that file marked as covered/uncovered?
Â
Ann
Yes, we pass in a coverage report, but none of the lines in the new file are marked as covered or uncovered. It’s almost as if the fact that there is no unit test for this class at all makes it omit the class from the coverage report.
Sigh. The package I added the class to is in an exclusion group. I need to move it.
Ok, after moving the class to a package that is not excluded, and adding many more untested lines to the class, it is STILL not failing the quality gate, even though both the coverage on new code and the coverage after merge are below the requirement of the quality gate.
David,
Based on the screenshot of your Quality Gate, all of your Quality Gate conditions are set on your Overall Code (meaning they aren’t being evaluated on the short-lived branch or pull request).
What if you add a Quality Gate condition for Coverage on New Code?
Colin
I don’t currently have the ability to edit the quality gate, but I’m asking about that. I see nothing in the view of the quality gate that says anything about a distinction between new or overall code.
Something we improved in 8.0 
For now, consider quality gate metrics with the word “New” as applying to new code, and those are the quality gate conditions applied to short lived branches and pull requests in 7.9
This is what I see for our quality gate, in 7.9.1:
So you’re saying that the fact that we don’t see “New” anywhere here means these are all for overall code? Do I even have the ability in 7.9.1 to set conditions for new code? If that was the case, that effectively means that in this version quality gates in pull request scans are useless.
Yes. You just add metrics like “Coverage on New Code” to the quality gate.
Ok, I now have edit rights to the quality gate, but this results in additional questions. The “Coverage” settings are straightforward, but I see some oddities with the conditions around “issues”. For instance, our quality gate currently has “Blocker Issues”, “Critical Issues”, and “Vulnerabilities”, implying that those are for overall code, not for new code. Curiously, when I look in the new conditions list, I see both “Coverage” and “Coverage on New Code”, but I only see “New Blocker Issues”, “New Critical Issues”, and “New Vulnerabilities”, but not the choices without “New”. Do we now not even have the ability to add a condition for issues in overall code?
Those conditions are already in your quality gate, so I imagine they aren’t available to add a second time
That would be logical, except for the fact that “Coverage” is also in the list of conditions, and I already have “Coverage” in the quality gate.
I can’t reproduce that behavior on my end.
How have your Pull Request Quality Gates behaved since adding the Coverage on New Code metric?
