Hi All,
This might sound bit basic but I couldn’t find it anywhere else.
My aim is to reject a Pull Request if it fails the Sonar Quality Gate. This works well for the first build of the PR where it analyze the modified files and make the Quality Gate as Failure, however if I re-build the same PR from Jenkins it doesn’t check the over-all files modified in a PR but it takes only for that particular build and since it doesn’t have any new files it make the Quality Gate as Success which obviously enable the PR to be merged to ‘master’.
Could someone through some light if I’m missing anything? or it is not the right approach for the Quality Gate status
To be clear, you’re using the Jenkins rebuild(?) feature to re-run the job on exactly the same commit? And if so, would you mind sharing why you find that to be useful (particularly if you got all the way through to the point of marking the QG status the first time)?
TBH, I don’t think this is a use case we anticipated.
Coverage on New Codeis less than 80.0%
New Bugsis greater than 0
Technical Debt Ratio on New Codeis greater than 5.0%
I’m expecting the same Failure on PR builds as well, even if I do any subsequent clean commits.
Instead, if I commit a 2nd file with clean code to the branch the PR becomes Green which obviously letting the defecting code to be merged to ‘master’. Please let me know if I’m missing anything?
And, I’m expecting my PR to get failed because one of the Run was Red, but the PR build is success and the code which showed error in the ‘feature’ branch doesn’t reflect with the PR build though it is listed under the modified files list
The first and third conditions are about ratios, so it makes sense to me that adding a new (“better”) file to the PR could change the ratios enough to make those conditions pass. For instance, let’s say file1 adds 5 uncovered lines: 0% coverage on new code. Then file2 adds 20 covered lines: 80% coverage on new code (20 of 25 lines covered). The thing that’s not easily explained away is the failure and then success of the New Bugs condition. Can you verify that the issues raised in the first commit were still open after the 2nd commit?
“The thing that’s not easily explained away is the failure and then success of the New Bugs condition”
Could be this due to the below status of the QG of the 2nd file commit?
“Some Quality Gate conditions on New Code were ignored because of the small number of New Lines”
“Can you verify that the issues raised in the first commit were still open after the 2nd commit?”
Yes - first commit and second commit are completely new files, so the first issue is still open.
FYI - I’m committing files on a test basis not with a real-time scenario
Now to dig into them…
If the issue is no longer reported in the PR, where did you go to make the screenshot? Was it taken earlier and you had it in storage? Or did you go to a different branch/PR? (Or just re-create the conditions?)
The interesting thing about this particular issue is that it’s entirely possible that a change in a different file could make it go away. As an example
/* File = A.java */
public Foo getFoo() {
if (condition) {
return new Foo();
}
return null;
}
/* File = B.java */
private void doTheThing() {
Foo myFoo = this.a.getFoo(); // might be null!
Bar aBar = myFoo.getBar(); // Could be an NPE; issue raised!
}
So let’s say the first commit adds doTheThing in B.java => NPE issue raised. Then the second commit makes a change in A.java:
/* File = A.java */
public Foo getFoo() {
if (condition) {
return new Foo();
}
return oldFoo;
}
And B.java’s issue goes away without it having been touched.
Screenshot are correct - PR doesn’t report any issue but it has the two files which are modified as part of the ‘feature’ branch which as reported as failure during branch analysis (refer the one of the images above please).
I think I will have to try it out with a real time scenario to get a better picture - main thing which I want to confirm is I’m not missing any configs during PR analysis.
