I’ve found that sonar.qualitygate.wait fails the build if the quality gate is broken - what IMHO is missing is an option to wait for the analysis to be finished (what exactly I was expecting the qualitygate.wait to do and in fact does) but without breaking the build (similar to maven.test.failure.ignore=true.
What I would like to do is to later in the integration build report what the result of the QG was (so it would be ideally added to the report.json, or at least having enough information in there to query the Sonar API for that analysis for the result of the compute engine task / quality gate) and mark the build as unstable rather than failed.
Something like sonar.qualitygate.ignore.failure would be nice to have for that.
The whole point of sonar.qualitygate.wait is to fail the pipeline if the project fails its Quality Gate. If you want to do something else, you’ll need to script it by hand. You should be able to get the ID of the CE task from the files SonarQube leaves in your workspace.
Well in general I’d agree but there are a few - let’s call them - edge cases.
First, the quality gate is for the overall project and if you do pull request analysis you end up with failing PR, because the project overall is in a bad state - not ideal, as the analyzed PR can be perfectly fine and also increase the overall project state to be better than before, so ideally it should not fail due to some other parts being broken.
Second, the qualitygate.wait is the only way to make the plugin “wait” until the server has finished it’s analysis. So if I wanna script sth by hand, ideally I would like to know that the analysis has finished before querying for the result. Looping, waiting and pausing is not something that is easily done in scripts. Given that the check engine already has that code ready it seems to me like exposing it besides the qualitygate makes sense. I’m perfectly fine decoupling that from the qualitygate and calling it just sonar.analysis.wait or sth like that…
Personally I’d favor sth like that because I can then easily use that to report appropriatly on the Gerrit review - even blocking the commit from going in. The issue with failing already in the build step is that the post build plugins in Jenkins won’t even get called…
But if it’s deemed an unncessary feature we will find something else For now it’s setting the qualitygates to no-ops, so that they’ll always pass which is something I think is way worse…
Happy to discuss the details but are you absolutely sure? We’re on 8.9 Enterprise, so maybe something changed in 9.0. There is a “per project” option to make the qualitygate to always compare against a specific version / main branch - but by default the quality gate uses metrics either based on last X number of days or against the last version. And if for example the metric “coverage on new code” is below the threshold - the gate will fail. I have examples were the coverage even increased in the PR, but the resulting coverage on new code was still below the threshold so the gate still failed.
Also the compare to branch / reference is only available on project level not globally settable as a default, which would mean we’d have to adjust nearly 300 projects manually ()
Well, what I meant is the new code setting on project level. There we have 3 options. Last release / X days / Reference branch.
And as far as I can tell there is no difference if I analyse a branch or a PR, whatever setting on the quality gate is set regarding new code will be applied and if the gate fails already on a project then no matter what the change being analyzed does unless it fixes ALL issues the gate currently has - the maven build will fail.
Yes, ideally we wouldn’t have projects with failing gates but we do (actually with our even very moderate initial settings we have them failing on nearly 50% of the projects. For an initial rollout we need to start somewhere and unfortunately in order to get the Jenkins plugin to post results to our Gerrit reviews we need somehow to wait during the build until the analysis is done… The only way we found is the qualitygate.wait setting. So unfortunately we dropped all new code and also all overall code conditions on the default gate and I’m a bit afraid we will need a LONG time to get even moderate settings back because either we need to define dozens of quality gates and apply them individually as each project matures (very carefully checking that we NEVER ever cause a failing gate) or we need to wait until the very last project is ready until we tighten the conditions…
FYI, there’s actually a 4th option for the New Code Period: at a branch level (under the cog menu) you can choose a specific analysis as the baseline.
Regarding the Quality Gate in general, the point of having the default Quality Gate focus on New Code is that you should be able to meet the criteria in every project, greenfields or legacy. If every project is currently failing New Code criteria, I would take a look at the New Code period (is it too long?).
Ehm… This sounds a bit dire. If the root of all this is that you don’t want the pipeline to fail for a failing Quality Gate, then that’s easily done. If on top of that you want to report a failing Quality Gate in a successful pipeline, I guess you’ll need to do it the old way: with sleep and a loop.
Thanks, that was the point (it was initially posted as feature request) - the sleep and wait in a loop is already in the shared scanner code - it would just need to be exposed in the maven plugin similar to the Gitlab plugin. I thought it’s a useful extension there hence the post. I’m fine if it’s not accepted. Then we’ll have to live with our current solution and see if we want to “move” the wait to another place…
There someone implemented a QualityGateFailMode which can be set to WARN or even NONE. (
For now it’s setting the qualitygates to no-ops, so that they’ll always pass which is something I think is way worse…
)