Dotnet Core code coverage inconsistent

SonarQube Cloud
,
1

Hello,

We’re using SonarCloudAnalyze in our Azure DevOps CI pipelines. We’re getting very inconsistent results with the code coverage and have linked it to the output names of the trx / coverage files. It appears that the files are created with a name format that has a datetime suffix e.g.

build000GCC_2021-01-28_14_40_48.trx
build000GCC_2021-01-28_14_40_42.trx
etc.

Sometimes we seem to end up with multiple files with the same name except with an extra [1] - presumably because the tests completed at the exact same time:
build000GCC_2021-01-28_14_40_42.trx
build000GCC_2021-01-28_14_40_42.[1].trx

When this happens the following sonar step complains that it cannot find it:
None of the following coverage attachments could be found: build000GCC_2021-01-28_14_40_42.[1].trx

At which point the coverage drops to something like 50% of the regular coverage. When the filename are unique then it shoots back up.

If you look at the timeline of the code coverage it goes up and down daily (presumably because two tests completing at the same time is random).

We’re using:
dotnet test --logger trx --collect "Code coverage"
to generate the file.

Does anyone have any suggestions?

2

Hi @seanbrotherton and welcome to the community !

Can you check in the logs that the file with the [1] gets really generated by concurrent tests ? Or might it be a “Re-run failed tests” option activated somewhere ? Can you check also if you have any runsettings file that might have specific configuration leading to this ?

Mickaël

3

Hi @mickaelcaro

Thanks for the quick reply. I’m pretty sure its due to multiple projects all getting tests results complete at the same time. I’ve attached the logs of the dotnet test task - as you can see there’s 2 unit test projects and both run successfully with no failures (slightly different number of tests) but appear to complete at the same time.
dot-test-log.txt (5.1 KB)

We dont use any runsettings across any of the repo either :frowning:

Thanks,
Sean

4

Hello,

We’re still experiencing this issue. Is there anything else we could try or anything we can look into to investigate further?

Thanks,
Sean