Hi, i’m trying to understand how do you calculate the metrics for PRs, and especially code coverage. I always assumed for git, all measures are based on the changed lines (based on the blame info).
Right now I have a failing PR which only changed 4 lines in 2 files:
Looking at one of the files (the other is much smaller), it seems like all of the lines are calculated for the code coverage - not just the 4 changed lines:
So my question is - how do you calculate the metrics in general and the coverage metric in particular for PRs? shouldn’t it only take into account the actual line/s changed? or is it only working on the file level?
I got an even worse example - a developer deleted two lines inside an uncovered file, and the PR failed the quality gate because there was not enough (actually 0%) coverage! It doesn’t make sense - if anything, deleting code lines should make things better with regards to coverage…
First of all, we try to identify the changed lines using information from SCM. This requires:
The target branch (to merge into) is correctly specified with sonar.pullrequest.base
The target branch is fetched (otherwise we cannot compute the diff)
From the diff we identify the changed lines per file.
To compute the coverage, we take the coverage report, and keep only the lines we have identified in the diff. If the diff identified 4 changed lines, and 1 is not covered, that will give a coverage of 25%. Only the lines in the diff count, the size of the rest of the file doesn’t matter.
Looking at your screenshots, it looks like the computation of the diff is not correct. The prime suspect is the above two bullet points. Please check on that.
To investigate further, please send the scanner logs, and the actual line that triggered the analysis, if possible.
On top of what was said by Janos, please send the scanner logs with debug enabled.
Also, the scanner should log in debug the sha1 used when calculating the diff. You can check in your git repository whether it makes sense.
Sorry it took my a while, but i’ve validated it was indeed a configuration issue and not a product bug (however better wording might have helped).
I didn’t specify sonar.pullrequest.base at all because I thought the default value (master) is referring to the Sonar server “master” branch (can be seen in my second snapshot - “for merge into master from PR-453”) and not to the actual git master branch - in my case the default branch that my PRs are merged into is not called master in bitbucket/git, but since i’m scanning this branch without a sonar.branch attribute it is called master on the Sonar server.
Now I understand the sonar.pullrequest.base attribute is referring to the local scan git branch name and when I use the actual branch name everything works as expected.