Pull request branches to include code coverage

branches
coverage

(Peter Connor) #1

The PR branches which tell you instantly whether code to be merged passes the quality gate is an awesome thing and greatly helps us time-wise with conducting code reviews.

However it only applies to bugs, vulnerabilities and code smells, whereas our quality gate takes into account code coverage and duplication too.

Please could the small green dot take into account cc and duplication and also highlight what the measures for new code to be introduced are going to be,


Lines to cover on new code is 0 in PR and for branches
Quality Gate + Short Lived Branches
Is it possible that i feed the sonar two branches of one project and get the coverage report of the diff line?
(Fabrice Bellingard) #2

Hi Peter, this is in our short term roadmap: https://jira.sonarsource.com/browse/MMF-1118

So stay tuned!


(Peter Connor) #3

That’s great news, I know it will certainly help us a great deal. I guess if it is in specification now, it might be ready for the 7.4 version sprint?

Thanks,
Pete


(Fabrice Bellingard) #4

I can’t give you an ETA yet (it might indeed be in the 7.4 release), but this will come for sure since even here at SonarSource, our dev teams highly want this feature! :slight_smile:


(John Bateman) #5

I would love this feature!


(Michael J Vinca) #6

I also would appreciate this feature. Frankly, in our workflow, once something gets into develop or master (our long lived branches) our Quality Gates are already too late for the feedback loop. We want those gates to actually act like gates, and prevent feature branches from getting into develop. Quality Gates on features would be our primary use of the tool. Having a Quality Gate on develop would just be a nice-to-have backup.

Regarding the questions raised in MMF-1369 (linked to MMF-1118), I would agree that a feature branch would use the “… on New Code” settings of a Quality Gate. As far as the different definitions for different branches, I would put that in the nice to have category. Just having the Quality Gate apply to all branches and not just long term would be an excellent first step and enough for us. (i.e. I would prefer to get that, then more configurability later than to wait longer for a more configurable solution.)


(Peter Connor) #7

They still haven’t started developing this yet judging by the Jira boards which is a real shame. I think its the only thing missing from making the product truly excellent.

We use individual branches per developer and a leak period of previous version which is our sprint number. As developers work on different projects their leak periods have reset at different times and so checking the code coverage on their own branch isn’t an accurate reflection of what it will be on the master branch. This is the only cause of the master branch quality gate failing for us, as a pull request isn’t allowed to be completed without the quality gate passing.


(Peter Connor) #8

Good to see this is finally making an appearance in SonarCloud :slight_smile:


(Maxwell) #9

Are you saying the coverage is being taken in consideration to mark the branch/pr analysis quality gate as failed/pass on sonar cloud? I can’t see any change like that yet.


(G Ann Campbell) #10

You’re right, coverage is calculated but doesn’t yet affect a PR’s status. Why? Well, what’s enough? Your answer is likely to be different from the next guy’s. We’ll get to this eventually when we tackle “real” quality gates on branches.


(Maxwell) #11

Yes, having the quality gates integrated into the branch/pr analysis is the ideal, we looking forwarding for that, hope it will be ready soon. =)

Thanks Ann.


(Peter Connor) #12

I would expect it to be based on the rules set in your quality gate, so for us I would expect the quality gate to fail if the estimated code coverage is less than 80% for the leak period.

Could you also explain what the difference is between the two icons at the top of the PR?

As far as I assume, the left stat is what the total coverage will be on the master branch once the PR has been completed and I’m hoping that the right stat is the what the total coverage will be for the leak period once the PR has been completed.

If this is the case, can I ask why the second pull request doesn’t have the leak period statistic on it? I would expect to see this statistic on all pull requests.

PR-1

PR-2

Looking forward to the work to really lock down the quality gates :slight_smile:

Pete


(Nicolas Bontoux) #13

Hey Peter,

I do have to ask (and apologies if you did already): did you hover the small question marks next to the number ? :stuck_out_tongue: We’ve put them here precisely to help with the understanding.

Correct. Tooltip says: ‘Estimated Overall Coverage after merge.’

Tooltip says ‘Coverage on New Code. See Measures for details.’. Meaning that it corresponds to the coverage on the actual code modified/added in your PR, as detailed in the Measures page of your PR.


(Peter Connor) #14

Yes I read them, the left one is fine it was just more the right one I was having issue with.

Everywhere else on the site the peach coloured boxes are all related to leak periods (our leak period is ‘previous_version’), so this implies that would be the overall estimate for the leak period, which is ideally what we personally wanted from this feature (because of our quality gate settings mentioned above).

Coverage for the PR code and coverage for the leak period are far more valuable in my opinion that overall code coverage as that is easily available on the home screen of the project.

Just my 2 cents worth!

Thanks,
Pete


(Maxwell) #15

Would be nice if we could have the lines marked in the pr as well, just like they are marked when looking the code in sonar (red/green) based in the coverage, i have seen other tools doing that on github, so i think its possible somehow.


MMF-1118 Release Version?
(Peter Rockstars) #16

Awesome that this is making it’s way into Sonar. I am just wondering, the issue in JIRA is marked as Closed, does that mean that we should be seeing this in SonarCloud?

I’m asking because we still see “lines to cover 0” on new code in our PR’s and short lived branches.


(G Ann Campbell) #17

Hi,

Where are you seeing “lines to cover” on a PR? If you’re seeing it on the PR’s measures page, then you’re demonstrably already using the feature the the question to answer is why you’re getting zeros.

Here’s a PR on a public project picked pretty much at random that shows coverage in the header and coverage-related metrics in the rail: https://sonarcloud.io/component_measures?branch=drivers-on-bms&id=it.eng.knowage%3Aspagobi-metamodel-core

 
Ann


Quality Gate + Short Lived Branches
(Peter Rockstars) #18

Hi Ann, below is a screenshot where I would expect there to be more uncovered lines. It’s an entirely new class. When I open the class, I correctly see the red indicators that the code is not covered. But none of the lines are colored yellow/orange to indicate that it is new code, which is the case when we view a class with new code in master. This is from a long-lived branch but the behaviour is the same for short-lived branches.


(G Ann Campbell) #19

Hi,

Your header shows 39% coverage, so the value is calculated. The question is why it’s not shown in the UI for you.

At this point, I’d advise you to

  1. create a new thread
  2. include in it any errors in your browser’s developer console.

 
Ann


(Peter Rockstars) #20

Thanks, there are no issues in the console, but I’ll make a dedicated thread