Where should unit tests be shown

SonarQube
, ,
1

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)

    • Version: 10.2.1.78527
    • Date: 2023-10-18

  • how is SonarQube deployed: zip, Docker, Helm

    • Local

  • what are you trying to achieve

    • Displaying unit test information

  • what have you tried so far to achieve this

    • Importing ccpunit formatted information and clicking on the # Unit Test Link:

So I have cppunit test result data getting parsed into my project (see attached -X log)
sonar_x.log (38.1 KB)

I can see the 2 Unit Tests under the Coverage section

image
image754Ă—385 12.8 KB

If I click on the link from 2, I can see the unit tests on the left panel and above the src tree, as well as the See History. The History page displays well and I have no questions about that.

image
image1656Ă—327 13.7 KB

But when I drill down in the source directory, I don’t see any unit test information:

image
image785Ă—460 9.59 KB

The unit results file has locations to the of the file src/src_001.c and the line number:

<FailedTest id="2">
    <Name>test_002</Name>
    <FailureType>Failure</FailureType>
    <Location>
        <File>src/src_001.c</File>
        <Line>6</Line>
    </Location>
    <Message/>
</FailedTest>

So my question, should I be seeing unit tests be listed in the source tree and on the source display?

Thanks!

Tim

2

Hi Tim,

It’s important to understand that SonarQube doesn’t count your tests. If you feed your test files into analysis (with sonar.tests) then you’ll be able to see the files your tests are in in SonarQube. But SonarQube can’t identify the tests in those files for you.

When it says there are 2 unit tests, it’s merely taking your test execution report at face value and repeating its data to you.

Does that make sense?

 
HTH,
Ann

3

Yes…kind of, but a few follow on questions:

  1. I don’t see where sonar.tests exist. I assume that you mean sonar.test.inclusions. If that’s correct, cool
  2. The unit tests have paths to source files failed test. So you’re telling me that SonarQube doesn’t correlate unit test “location” (ie. source file not test file) with any of the analysis? If so, that’s okay, I’d like just to verify before I move on. src/src_001.c is a source file that get analysis and coverage mapped to it.
<FailedTest id="2">
    <Name>test_002</Name>
    <FailureType>Failure</FailureType>
    <Location>
        <File>src/src_001.c</File>
        <Line>6</Line>
    </Location>
    <Message/>
</FailedTest>
4

Hi,

It’s an analysis parameter.

No, actually. sonar.test.inclusions is used to specify a subset of what sonar.tests encompasses. The relationship between the two is discussed more fully here.

Uhm… I’m not sure I understand the question. What I can say is that SonarQube extracts pure numbers from your execution reports. And that’s all. We used to have more intricate functionality here, but dropped it at least partly because of our fundamental belief that test execution data should be dealt with before you get to SonarQube. (E.G. if your tests are failing… should you really move forward to analysis? Shouldn’t that fail the pipeline?)

 
HTH,
Ann

5
  1. Ok - i see it now. I was searching for it under the general settings
  2. Understood
  3. So you are just presenting the number of test cases, failed test case and error test cases. You don’t use the the Location tag in the CppUnit report and don’t tie the source listed with the source analyzed. I get that now.

Re: failing the pipeline on a single failed unit test - That’s not necessarily the case. I know on Jenkins - Junit, it uses Health report amplification factor (see below) that allows ethe number of failed test cases to set the build as unstable until it reaches a certain ratio, then it will fail the pipeline

Health report amplification factor: The amplification factor to apply to test failures when computing the test result contribution to the build health score. The default factor is 1.0. A factor of 0.0 will disable the test result contribution to build health score, and, as an example, a factor of 0.5 means that 10% of tests failing will score 95% health. The factor is persisted with the build results, so changes will only be reflected in new builds.

Suggestion: It would be nice to show an indicator on the code view for failed test case and hover over to get the a pop up info as with a “code smells” indicator. Just a suggestion for the future

SonarQube
SonarQube3843Ă—2191 384 KB

Thanks for your help!

Tim

1 Like