Pull request branches to include code coverage

coverage
branches

(Peter Connor) #8

Good to see this is finally making an appearance in SonarCloud :slight_smile:


(Maxwell) #9

Are you saying the coverage is being taken in consideration to mark the branch/pr analysis quality gate as failed/pass on sonar cloud? I can’t see any change like that yet.


(G Ann Campbell) #10

You’re right, coverage is calculated but doesn’t yet affect a PR’s status. Why? Well, what’s enough? Your answer is likely to be different from the next guy’s. We’ll get to this eventually when we tackle “real” quality gates on branches.


(Maxwell) #11

Yes, having the quality gates integrated into the branch/pr analysis is the ideal, we looking forwarding for that, hope it will be ready soon. =)

Thanks Ann.


(Peter Connor) #12

I would expect it to be based on the rules set in your quality gate, so for us I would expect the quality gate to fail if the estimated code coverage is less than 80% for the leak period.

Could you also explain what the difference is between the two icons at the top of the PR?

As far as I assume, the left stat is what the total coverage will be on the master branch once the PR has been completed and I’m hoping that the right stat is the what the total coverage will be for the leak period once the PR has been completed.

If this is the case, can I ask why the second pull request doesn’t have the leak period statistic on it? I would expect to see this statistic on all pull requests.

PR-1

PR-2

Looking forward to the work to really lock down the quality gates :slight_smile:

Pete


(Nicolas Bontoux) #13

Hey Peter,

I do have to ask (and apologies if you did already): did you hover the small question marks next to the number ? :stuck_out_tongue: We’ve put them here precisely to help with the understanding.

Correct. Tooltip says: ‘Estimated Overall Coverage after merge.’

Tooltip says ‘Coverage on New Code. See Measures for details.’. Meaning that it corresponds to the coverage on the actual code modified/added in your PR, as detailed in the Measures page of your PR.


(Peter Connor) #14

Yes I read them, the left one is fine it was just more the right one I was having issue with.

Everywhere else on the site the peach coloured boxes are all related to leak periods (our leak period is ‘previous_version’), so this implies that would be the overall estimate for the leak period, which is ideally what we personally wanted from this feature (because of our quality gate settings mentioned above).

Coverage for the PR code and coverage for the leak period are far more valuable in my opinion that overall code coverage as that is easily available on the home screen of the project.

Just my 2 cents worth!

Thanks,
Pete


(Maxwell) #15

Would be nice if we could have the lines marked in the pr as well, just like they are marked when looking the code in sonar (red/green) based in the coverage, i have seen other tools doing that on github, so i think its possible somehow.


MMF-1118 Release Version?
(Peter Rockstars) #16

Awesome that this is making it’s way into Sonar. I am just wondering, the issue in JIRA is marked as Closed, does that mean that we should be seeing this in SonarCloud?

I’m asking because we still see “lines to cover 0” on new code in our PR’s and short lived branches.


(G Ann Campbell) #17

Hi,

Where are you seeing “lines to cover” on a PR? If you’re seeing it on the PR’s measures page, then you’re demonstrably already using the feature the the question to answer is why you’re getting zeros.

Here’s a PR on a public project picked pretty much at random that shows coverage in the header and coverage-related metrics in the rail: https://sonarcloud.io/component_measures?branch=drivers-on-bms&id=it.eng.knowage%3Aspagobi-metamodel-core

 
Ann


Quality Gate + Short Lived Branches
(Peter Rockstars) #18

Hi Ann, below is a screenshot where I would expect there to be more uncovered lines. It’s an entirely new class. When I open the class, I correctly see the red indicators that the code is not covered. But none of the lines are colored yellow/orange to indicate that it is new code, which is the case when we view a class with new code in master. This is from a long-lived branch but the behaviour is the same for short-lived branches.


(G Ann Campbell) #19

Hi,

Your header shows 39% coverage, so the value is calculated. The question is why it’s not shown in the UI for you.

At this point, I’d advise you to

  1. create a new thread
  2. include in it any errors in your browser’s developer console.

 
Ann


(Peter Rockstars) #20

Thanks, there are no issues in the console, but I’ll make a dedicated thread


(Thomas A. F. Thorne) #21

Now that the request at https://jira.sonarsource.com/browse/MMF-1118 is marked as Fixed can you give an ETA on the release date or release number that will include it?


(G Ann Campbell) #22

Hi,

This was delivered in 7.4.

 
Ann


(Thomas A. F. Thorne) #23

Thank you. I shall as our DevOp or IT guys to try applying the update to our system so that we get the new feature! :smiley_cat:


(Roman Borysenok) #24

Hi Ann,

Great news! Maybe you know when it will be available in SonarCloud as well?


(G Ann Campbell) #25

Hi @barfet,

This was available in Sonarcloud before it was available in SonarQube.

 
Ann


(Roman Borysenok) #26

Hi @ganncamp,

Thank you for your reply.
Maybe you could clarify this one.
Currently we’re using SonarCloud with Azure DevOps and set it up for master and feature branches.
On each PR we trigger build + SonarCloud check (approve from SC is required).
But currently SC doesn’t count a test coverage metric in quality gate for short lived branches (PR) and you could merge a PR without passing a test coverage check.
I thought that we’re talking about this metric and you add support for this kind of scenario (block PR, for example, if test coverage is less that 80% on new code). If we’re talking about the same staff, maybe you could suggest how we could set up it in SonarCloud? Because we don’t have such functionality i a moment.

Thanks!


(G Ann Campbell) #27

Hi,

Sorry to be pedantic, but reading your coverage report and applying a coverage criteria in a quality gate are two different things. 7.5 will add duplication calculation on PRs and SLBs, and hopefully 7.6 will add “real” quality gates.

And before you start to yell, please check the other threads in this community where everyone else has already yelled. Feel free to click hard when you ‘like’ those posts.

 
Ann