How to removed rules in a new plugin version

Is it possible to remove old rules from a plugin? What happens if they are used in some quality profile?
I tried doing that but SonarQube throws org.sonar.server.exceptions.BadRequestException: Rule was removed during startup.

What is the best practice for this?
Keep the rule around forever and deprecate it?