We are in the process of evaluating SonarCloud in comparison to our current, on-prem SonarQube and am having issues with request analysis. Specifically with inline comments.
The analysis completes successfully and the Pull Request check shows up in GitHub (with a failed status), however I do not see the inline comments. Is that a configuration option or have those gone away in SonarCloud?
Indeed we have removed all comments from the Conversation tab. See this announcement.
@janos That seems foolish. That was one of the biggest selling-points of using SonarCloud vs other scan services when we did an analysis last year.
Can we expect the Sonar-injected comments in the Conversations tab to come back in the future?
It seems the Conversation tab is intended for content by humans, not by robots. For that reason we don’t have plans to go back on that route. It seems Checks is the way to go for such automated checks that we do. We are considering to enrich the feature in that direction.
Although I appreciate that conversations are better for humans, there is already extensive precedence for robot actions being annotated in that view. Furthermore, it is of massive value to get an automated GitHub “Review” from SonarCloud, and it would be good to use GitHub’s functionality for displaying that. This seems to have been taken away.
It is extra frustrating as I had sunk some time into trying to get this work as is described in your documentation:
Pull Request analysis allows you to:
- see your Pull Request (PR) analysis results in the SonarCloud UI and see the green or red status to highlight the existence of open issues.
- automatically decorate your PRs with SonarCloud issues in your SCM provider’s interface.
https://sonarcloud.io/documentation/analysis/pull-request/
However, since your announcement, this is not how it works. At least please update your documentation to make this more clear.
And, please reconsider this decision.
As @janos said, Checks functionality is now the way GitHub expects external integrations like SonarCloud to contribute information to a Pull Request - instead of “abusing” the comment feature to kind of “pollute” human reviews.
If you think this is not working well, I suggest that you push this feedback to GitHub forums.
Can you clarify what makes users think that the PR decoration is done through comments instead of checks? The documentation does not mention comments - so I’m a bit confused.
I believe this line is misleading.
- automatically decorate your PRs with SonarCloud issues in your SCM provider’s interface.
It certainly led me to believe that the GitHub plugin operated in the previous way where each issue showed in-line with the code.
I looked into the GitHub checks feature to see if I should complain to them, but I shouldn’t as GitHub Checks already supports “Annotations” which achievs this. Their Hello World example shows this: Building CI checks with a GitHub App - GitHub Docs
Have you migrated to GitHub annotations, or is it an error in my configuration that means I am not seeing annotations/inline decorations?
I guess you might have missed the following part in PR analysis results are now displayed in the Checks tab of Pull Requests in GitHub:
As you will see, we decided for now to not display issues that are discovered in the code of the Pull Request (as well as issues that are not in the code).
The reason is that with Checks, we first wanted to use Annotations to display issues. We realised that we would not be able to display in the best way we want the issue information such as the type of the issue (bug, code smell, vulnerability), the severity of the issue and the link to the issue in SonarCloud. You can still retrieve all those information on SonarCloud, via the link.
Correct, I missed that.
That only makes me more disappointed that you thought about it and chose not to use it. This has quite a negative impact on our workflow.
Like with @Rick-at-KUBRA, this is a key function for us and was part of our selection criteria when we migrated from Codacy. This change puts us in a tough position.
I’d like to dig into why you think this puts you in a tough position. With the quality gate summary and the links available in the SonarCloud Check page, you get the big picture of the quality of the PR and can get access (through the links) to the complete list of found issues with far more details, explanation and help than what you could get with annotations inside GitHub.
@Fabrice_Bellingard I think a simple way to explain it is that ideally you want your developers to stay in a simplistic workflow. I went to Atlassian’s talk at AWS Re:Invent 2 years ago where they explained how they thought you should use github hooks to push Jira tickets to sequential states so that a developer doesn’t need to interrupt coding to go do that in Jira as a secondary task. In my mind, this is a similar situation where if you tell a smart developers “Hey, there’s a problem in this line of code!”, they may easily identify and fix the problem without the extensive details, explanation, and help at sonarcloud.io and can then go back to work.
So ideally the plugin would use annotations in the files tab to point to where an issue is occurring AND provide some links (in the annotation or the Checks tab) to help green-er developers get further help in solving the problem.
I think @Rick-at-KUBRA captures the argument well.
It’s fine if you want to move the feedback to the Checks tab, but please add the inline annotations to the check. Even if you can’t format those annotations exactly as you might like, they could still make a huge impact in developer productivity.
(FWIW, this is a significant factor in our evaluation as we choose a tool to adopt.)
FYI, we’ve been discussing the topic internally for quite some time already. I hope we can find a way to add GitHub Checks Annotations while overcoming their limitations and still keep a good user experience. We’ll keep you posted.
Thanks all for your feedback.
Just wanted to chime in that our team was HUGELY disappointed to find that the inline annotations were gone. This was a major selling point, and the result is that people now basically don’t know or see if they’ve had violations.
I don’t mind having the summary results in the Checks tab, but missing the inline annotations for individual violations is a very significant regression for our PR workflow.
This is something we’ll be adding hopefully soon.
We are currently evaluating to move from our self hosted SonarQube (with the GitHub plugin) to SonarCloud.
We were quite disappointed that the comments are gone. Back in the time before we used the comments we noticed that most developers were not taking much interest into the scan results, only if there were severe errors. Once we had the comments, the people started to fix much more issues and showed a lot more interest.
We are very interested in the inline annotations! Is there some kind of roadmap for the SonarCloud customers available?
Update on this thread.
We’ve recently spent some time looking again at this (MMF-1535), and we were again disappointed by the poor user experience that those annotations bring. By the way, GitHub Checks and its annotations are still declared as “Preview features [that] are not supported for production use”: this is not surprising that no code quality service is currently relying on annotations.
There is a very powerful yet simple way to force developers to take care about issues found by SonarCloud: you enforce a green SonarCloud status on the PR to allow the merge.
Hi Fabrice,
Thanks for the update.
I think we’d get a lot of resistance on this; I myself wouldn’t enjoy it. We use SonarCloud to make our code review process easier. It was great when it was putting inline comments on PRs, basically saving reviewers tons of work commenting on basic things so we could focus on overall design. Also, code smells are not always blocking the quality gate (Log in with Atlassian account) but we still want people to see them.
Currently as a reviewer I have to use both the GitHub and SonarCloud UIs, and most reviewers don’t remember the SonarCloud part.
(Also, not every issue Sonar finds is real, and I don’t want to force developers to go outside GitHub to click “false positive” or “won’t fix” on every issue.)
We really miss the old behaviour where SonarCloud would put inline comments on code issues. (The only painful part of it was it was doing each comment individually, so we’d get dozens of e-mails for each PR instead of a single e-mail with all comments. If there’s a batch API it would probably improve things.)
I’m sorry to hear the checks/annotations API doesn’t provide a good experience. Unfortunately, without inline PR comments of some sort, SonarCloud is barely helping us at all. It’s too hard to have two streams of code review (GitHub + SonarCloud UI).
Best,
Mike
I was curious so I took a look around at what some other code quality tools do with GitHub PRs.
Hi Michael, thanks for your inputs on this. To be clear, we would prefer to be able to use annotations.
If you know other code quality online services which use annotations, I’d be really happy to take a look at what they do.
