Unit tests results are overriden

SonarQube
, ,
1

Must-share information (formatted with Markdown):

  • Sonarqube Community Edition v10.7 (95233), SonarScanner 5.0.1.3006
  • Docker
  • Trying to have all unit tests reported in Sonarqube

I’m trying to import different unit tests to Sonarqube. I have basically two different kinds of unit tests, ones that “links” to a file and then ones that doesn’t actually “link” to any file. Problem is that if add all of them to the analysis the ones that links to a file “overrides” the ones that doesn’t actually link to any file. If I add the tests that doesn’t link to any file, they will show fine in the SQ UI but with both enabled only the linked ones are shown.

EDIT:
Example:
sonar.junit.reportPaths → links to a resource

sonar.python.xunit.reportPath= → doesn’t link to a resource
(sonar.python.xunit.skipDetails=true enabled)

Without junit shows those python xunit results in the UI but with junit enabled doesn’t show them. Only if those python xunit results links to a resource it will show both.

2

Hi,

Welcome to the community!

Could you double-check your SonarQube version? 10.7 hasn’t been released yet. :sweat_smile:

Could you also expand on what you mean by “doesn’t link to a resource”?

This is a little bit of apples and oranges. Which language(s) is your project in?

 
Thx,
Ann

1 Like
3

Hi!

I mean by the “link to a resource” that in the SQ UI (measures/tests/Unit Tests) it shows the number of tests per file e.g.

image
image984×183 7.07 KB

And when it doesn’t “link to a resource” it look like this in the UI:

image
image987×181 7.17 KB

When I add the these 2 types of unit tests to a single analysis, only the ones that “link to a resource” will show up in the UI (189). But I want 189 + 1101 in the UI. (Mainly want to see the possibly failed as a number for quality gate)

We have multi language project (C++, Python, Java).

4

Hi,

Your screenshots are of a directory/file-based presentation. I’m guessing we’re looking at the Measures page / Unit Tests here?

The page is designed to help you understand how the measures relate to the project structure. E.G. How much complexity in this module? Which directories / files aren’t covered by tests? And so on.

You’re not seeing the count here because there’s nothing to “hang” it from.

 
HTH,
Ann

5

Yes i know that but the problem is that I can not see the “unlinked” unit tests anywhere if I add tests that “links” to the analysis. I would want to see the total of unit tests somewhere (especially failed/error tests):

image
image319×854 21.2 KB

The problem is that those +1000 unit test metrics go missing if I add tests that actually “links” to a file.

6

Let me try to explain it more clearly:

I send Java unit tests to Sonarqube (Ones that point to an actual file through the XML file = “links” to a resource) with sonar.junit.reportPaths.

It will look like this in the UI:

image
image560×1126 34.3 KB

Then I wanted also send Python unit test results to Sonarqube (they do NOT “link” to a file inside the XML file) with sonar.python.xunit.reportPath and sonar.python.xunit.skipDetails=true.

It will look like this in the UI (Without Java unit tests enabled):

image
image562×1041 30 KB

But if I have both enabled (Java unit tests and Python unit tests) in the same analysis it will look like this in the UI:

image
image560×1126 34.3 KB

→ Python unit test results go missing.

I tested manually adding the classname to the Python unit test XML file (points to actual file) and then it showed up in the UI together with the Java unit tests.

7

Hi,

Thanks for the detail. I think I understand now.

With Python alone, and, per the docs

sonar.python.xunit.skipDetails=true to collect only project-level details.

You get those project-level details.

But with a mix of project-level and file-level (from Java), the project-level numbers are discarded in the math.

I’m going to flag this for the team.

 
Ann

1 Like
9

This one seams to be similar: