You can remove any comments from being added whatsoever by removing the DevOps Platform Integration in your project settings.
Restricting permissions would result in an analysis warning on each pull request because it was unsuccessfully decorating the PR.