The Code Coverage reports are working great for new code. But it doesn’t seem to show how a PR might impact code coverage elsewhere in the repo.
For example, suppose a PR deletes a bunch of tests. This would decrease the overall repo code coverage significantly. But because the places where code coverage decreased weren’t part of the new code, they aren’t being detected in the Sonarqube coverage reports.
I’m wondering if this is how things are meant to work with SonarQube. And whether or not there may be a workaround to show that information in the PR reports.
Yeah… we don’t show impacts of deleted stuff because there’s nothing to attach it to. PR analysis is strictly limited to what shows up on lines in the new version that were changed. Deleted lines aren’t in the new version.
Similarly, if you, E.G., delete the only use of a private variable, a legitimately new issue shows up (after merge) on the old declaration of the variable. That’s always a fun one to explain.
I’m going to flag this for Product Manager attention.
Thank you for your insight.
This is a very interesting topic. We know that PR analysis regarding code coverage is not perfect and that it can be improved. We take into consideration your feedback, this is not in the short-term roadmap but we are tracking the requests on this topic and are actively monitoring this thread.
It is definitely a challenging problem, largely because there are many potential causes of indeterminacies in code coverage for a large repo (async logic, etc.). It can take a fair amount of effort to get useful, clean data from that kind of analysis against a large repo. But, once it’s working it’s really powerful. I had assumed that SonarQube provided that kind of information. There are other available solutions which provide that analysis, but it can still be a challenge to implement, due to the indeterminacy issue.
Just throwing my 2 cents in, that if you guys could solve that problem and support this kind of downstream analysis for code coverage, that could provide a lot of value, especially for larger companies with larger repos.
We’ll keep a record of this and continue to look out for similar requests. We’ll update this thread if anything changes; in the meantime, you can see a selection of ideas we’re considering in our roadmap page.
