Why don't Unit Test failures show up in short term branches?

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • what are you trying to achieve
  • what have you tried so far to achieve this

I’m using SonarQube v7.9.2, Scanner v4.2.0.1873, and the SonarGo plugin v1.6.0 (build 719)

I want to see the Unit Tests, Unit Test Failures, Skipped, Duration, etc… on short term branches. I’ve confirmed that my Go report is being ingested properly because all of the Tests measures show up in my Main branch and any Long-lived branches. Short-lived branches (such as those that are created from a PR) are missing these measures though they run the same build and scan process.

I see a new feature request here (Unit Test Conditions on New Code Quality Gate), but I have not seen anything in any docs that state that these measures shouldn’t show up in short-lived branches or any explanation as to why they aren’t.

Is there any confirmation that this behavior is by design or explanation as to why this isn’t working today?

Hi,

In SonarQube 8.1 we dropped the Short-lived branch concept. Now they’re all just branches, with all the same measures. You might want to upgrade.

 
Ann

Yes I read about that and its on my todo list to do some testing with it, though I have some concerns about upgrading after reading this: Sonarqube 8.1 branch analysis, definition of new ?!

But since 7.9.2 is the LTS release I was still hoping to get an answer to my question as to whether this behavior is by design and intentional? Is this described any any of the online docs?

Hi,

Short-lived branches and PRs only report coverage and duplications, IIRC.

 
Ann

I have to say the more I look at this the more it seems like a bug in 7.9.2 to me. I discovered that even on a short lived branch if I go to a specific file where there are unit tests and click on the show measures on that file you see the unit test execution measures, they just aren’t showing up in the measures from the main screen. Here is the best screenshot I could grab of this:

Hi,

Again, short-lived branches have gone away in 8.1. You need to upgrade so you can get all your metrics on all your branches.

 
Ann

I am interested in upgrading but have not had a chance to investigate the impact of this issue in our environment: Sonarqube 8.1 branch analysis, definition of new ?!

In any case if 7.9.2 is LTS so shouldn’t reported bugs be addressed in 7.9?

This issue occured only in Sonarqube 8.1 with the redesign of the branching feature,
from short-lived / long-lived branches to a branch is a branch (with full dashboard and metrics).
So it’s no bug in the current 7.9.2 LTS version.
Unfortunately it keeps me from updating to Sonarqube 8.1

Hi,

We don’t consider this a bug; it’s working as designed.

 
Ann

that why i was asking for a reference to the docs indicating this was the intended behavior of short lived branches. it doesn’t make sense to me that the unit test success and failure info would be available in the short term branch (as depicted in the screenshot i offered) and then just arbitrarily not displayed in the branch measures and used in the quality gate. i’ve searched everywhere i can think of and i haven’t found anything stating that was intended behavior.

Hi,

There’s nothing in the docs about this. Our oversight, I guess. In case you’re interested in the history/why of that: when SLBs were first introduced they only reported issues. It was a Really Big Deal when we added Coverage and Duplications measures (and only those measures). That’s why I keep saying that if you want all measures, you need to upgrade.

 
Ann