Hi,
We are trying to use CircleCI to run the analysis in Sonarcloud instead of the automatic analysis, however, our desire, is that if the analysis is not a success because it doesn’t meet the quality gates specified, the pipeline run fails.
We are not able to do that right now, even if the analysis fails (and we can see it marked as a fail) it returns a success to CIrcleCI.
How can we do that? When we used automatic analysis if it failed, the PR failed.
- ALM used: GitHub
- CI system used: Circle CI
- Scanner command used when applicable (private details masked): Followed CircleCI tutorial
- Languages of the repository: Typescript and Javascript
- Error observed: Analysis fails as it doesn’t reach the 75% coverage expected, however, it returns error code 0 to CircleCI
Hello @andreummaestre,
We use GitHub checks to report a failed quality gate to a PR. This should work with both Automatic Analysis and CircleCI.
Do you not see a SonarCloud check being reported to your PRs now?
Hi Tom,
So the issue is we are not using Automatic Analysis.
We are using CircleCI integration, and what’s happening, is that yes, the Sonarcloud analysis fails because it doesn’t fulfill the quality gates for that project, however, the Sonarcloud orb is returning an error code 0 to CircleCI and even if the sonarcloud analysis fails, the step in the CircleCI pipeline is marked as a success.
Right now we are bypassing/fixing it with our own orb, calling the Sonarcloud API to ask for the result of the analysis and if it’s FAILED we fail the pipeline.
But we want that to be something that happens automatically.
We use the Sonarcloud orb, it runs the scanner, it analyses the new code, is the analysis a FAIL? Kill the pipeline. Is it a PASS? Keep going.
We don’t offer the functionality to fail the CircleCI pipeline currently, but we do have alternatives depending on your use-case.
However I don’t fully understand your use-case. Could you further explain what you want to achieve by failing the pipeline. Is to be notified of a failure, is it to block a PR from being merged?
It is to block a PR from being merged.
I know we could do this using the automatic analysis tool, however, that tool does not support coverage files, something we need and want in our projects, therefore, we are forced to use CI.
Since we are using CI and not the automatic analysis, we want Sonarcloud to be a quality gate in itself for the PR, so if the analysis does not pass, we want to kill the pipeline so the PR is blocked.
Like I said we should report a GitHub check from both Automatic Analysis and analysis from a CircleCI pipeline. So even though the CircleCI pipeline does not fail, your PR should still have a failed check like this:
Can you confirm that this is not the case for you at the moment?