No discovery of xUnit python unittest results created via unittest-xml-reporting

SonarCloud
, , , ,
1

We are running python unit tests via the unittest-xml-reporting (aka xmlrunner) package.
This outputs XML files in the xUnit format.
Based on the docs, SonarCloud supports xUnit files by setting the option sonar.python.xunit.reportPath.

We run tests with this command:

coverage run -m xmlrunner discover tests -o test-reports

In the test-reports output directory we see the expected test outputs e.g.:

test-reports/
├── coverage.xml
├── TEST-test_compare_utils.AnyMatchTestCase-20230118094324.xml
├── TEST-test_compare_utils.LenientCompareTestCase-20230118094324.xml
├── TEST-test_function_utils.IdentityTestCase-20230118094324.xml
└── TEST-test_list_utils.EnsureListTestCase-20230118094324.xml

Here a shortened test output as example:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="test_list_utils.EnsureListTestCase-20230118094324" tests="5" file="test_list_utils.py" time="0.003" timestamp="2023-01-18T09:43:24" failures="0" errors="0" skipped="0">
	<testcase classname="test_list_utils.EnsureListTestCase" name="test_dict_to_list" time="0.001" timestamp="2023-01-18T09:43:24" file="tests/test_list_utils.py" line="27">
		<!--Test that a dict is wrapped as list.-->
	</testcase>
</testsuite>

And we run sonar scanner like this:

sonar-scanner -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=ourorg -Dsonar.pullrequest.provider="BitbucketCloud" -Dsonar.tests="tests/" -Dsonar.projectKey=aprojectkey -Dsonar.python.coverage.reportPaths=test-reports/coverage.xml -Dsonar.python.xunit.reportPath="test-reports/TEST-*.xml" -Dsonar.sources=. -Dsonar.c.file.suffixes=- -Dsonar.cpp.file.suffixes=- -Dsonar.objc.file.suffixes=- -Dsonar.python.version="3.7, 3.8, 3.9, 3.10" -Dsonar.exclusions="tests/**/*,setup.py"

Relevant here the option:

-Dsonar.python.xunit.reportPath="test-reports/TEST-*.xml"

But SonarCloud does not scan or detect them and instead shows a warning:

No report was found for sonar.python.xunit.reportPath using pattern test-reports/TEST-*.xml

We don’t see what we are doing wrong. Help would be appreciated.

4

Hi @Spen ,

Indeed, it looks like the scanner is not able to find those test-reports.
Could you please activate debug log running sonar-scanner -X ... and send us the full scanner logs?

6

The only relevant log lines after enabling debug logs:

08:38:23.721 DEBUG: Using pattern 'test-reports/TEST-*.xml' to find reports
08:38:23.776 WARN: No report was found for sonar.python.xunit.reportPath using pattern test-reports/TEST-*.xml
08:38:23.776 INFO: Sensor PythonXUnitSensor [python] (done) | time=55ms

Doesn’t seem very informative.

7

@Andrea_Guarino We checked everything again and the files are there, they are just not picked up.

How can we open a ticket / bug report?

9

Hi @Spen ,

Sorry for taking long time replying.
I cannot reproduce the problem: I created a demo project somewhat similar to your configuration.

Could you please double check test-reports are actually generated by xmlrunner in your CI machine?

Hope it helps,
Andrea G.