Test execution partially ignored

Hi,

I have been using Sonar for years now without any issues, a great pleasure… until last week where I have encountered a really strange issue and I have absolutely no idea what is happening.

I am using the pipelines of GitLab to execute some separate unit tests and functional tests, and then merging the 2 lcov and 2 xunit reports for Sonar, which give me at the end 1 lcov file listing the code covered by both unit and functional tests, and 1 xunit file with all executed tests.

Then I pass these 2 files to our SonarQube 8.6 - Developer Edition with the sonar-scanner 4.5.0.2216.
Here is the sonar-scanner parameters I am using :

sonar.projectKey=xxx
sonar.projectName=xxx
sonar.projectVersion=$CI_BUILD_ID
sonar.sources=src
sonar.tests=test
sonar.javascript.lcov.reportPaths=test-results/lcov-merged.info
sonar.testExecutionReportPaths=test-results/xunit-merged.xml

Here is the content of the final xunit-merged.xml file after merging both unit and functional reports.

<testExecutions version="1">
<file path="test\unit\index.test.ts">
<testCase name=": Save" duration="4"/>
<testCase name="Get functional logs: with empty query" duration="3"/>
<testCase name="Class: ManagementFormService: Function: get (GET handler)" duration="4"/>
<testCase name="Class: ManagementFormService: Function: update (POST handler)" duration="4"/>
<testCase name="Module: Export management form service, controller and externally injected config service" duration="29"/>
<testCase name="Class: ManagementFormService: Function: createForm" duration="1"/>
<testCase name="Class: ManagementFormService: Function: save (success) and createForm" duration="1"/>
<testCase name="Get functionals logs by origin: From valid queries and request" duration="2"/>
<testCase name="Get functionals logs by origin: From valid queries but no request" duration="4"/>
<testCase name="Get functionals logs by origin: From valid queries and request" duration="1"/>
<testCase name="Get functionals logs by origin: From valid queries but no request" duration="1"/>
<testCase name="Get functionals logs by origin: From valid queries and request" duration="0"/>
<testCase name="Get functionals logs by origin: From valid queries but no request" duration="1"/>
<testCase name="Module version: Cache ON, Repository ON, Amqp ON: From valid UUID and request" duration="1"/>
<testCase name="Module version: Cache ON, Repository ON, Amqp ON: From valid UUID but no request" duration="0"/>
<testCase name="Module version: Cache OFF, Repository ON, Amqp ON: From valid UUID and request" duration="0"/>
<testCase name="Module version: Cache OFF, Repository ON, Amqp ON: From valid UUID but no request" duration="0"/>
<testCase name="Module version: Cache OFF, Repository OFF, Amqp ON: From valid UUID and request" duration="1"/>
<testCase name="Module version: Cache OFF, Repository OFF, Amqp ON: From valid UUID but no request" duration="1"/>
<testCase name="Module version: Cache OFF, Repository OFF, Amqp OFF: From valid UUID and request" duration="0"/>
<testCase name="Module version: Cache OFF, Repository OFF, Amqp OFF: From valid UUID but no request" duration="1"/>
</file>
<file path="test\functional\index.test.ts">
<testCase name="Boot: Boot and log booting status" duration="163"/>
</file>
</testExecutions>

It contains 22 tests, 21 unit tests executed by one test file (test\unit\index.test.ts) and 1 functional test executed by another test file (test\functional\index.test.ts).
The issue here is that Sonar only count the 21 unit tests and ignore the last one (as you can see in the capture).


The last, functional test, is ignored while clearly present in the xml.

In the exact same way, the code coverage is reduced. All the source code covered by the unit tests are marked as covered, but the source code covered by the functional test is marked as uncovered, while the lconv-merged.info clearly indicate that the file has been tested correctly.

As you can see in this capture, the lines 45, 46 and 49 of the file src/technical/technical.controller.rest.ts are marked as uncovered.

While the lconv-merged.info report clearly indicate that these lines are covered.
Screenshot_20210129_225627

If Sonar was saying there is no test and no coverage, I would thing about something malformed in the report files. But here it’s like Sonar is cherry-picking what it want to display and I don’t know why.

All the source codes are in the same src source path, accessible by sonar-scanner, and browsable in sonar (which indicate they habe been analyzed correctly).

I have been searching for a full long week now about why would be a part of both reports be ignored while taking the rest of the files, and I still have absolutely no idea.

Can someone help me ?