Template for a good bug report, formatted with Markdown:
versions used: latest sonarcloud 1.5.1
error observed (wrap logs/code around triple quote ``` for proper formatting)
the plugin looks for trx file under the “wrong” path (a different path than what is actually used).
I would expect the plugin to continue to check for trx file in the same (which seems “correct” according to docs) path but also under “agent temp” directory where the trx and coverage file seems to be saved currently.
it seems the VSTS team recently change test output directory se: vstest output path changed recently · Issue #7975 · microsoft/azure-pipelines-tasks · GitHub
2019-01-02T21:51:11.2305659Z 21:51:11.229 Fetching code coverage report information from TFS...
2019-01-02T21:51:11.2321799Z 21:51:11.231 Attempting to locate a test results (.trx) file...
2019-01-02T21:51:11.2919665Z 21:51:11.291 Looking for TRX files in: D:\a\1\TestResults
2019-01-02T21:51:11.2923403Z 21:51:11.291 No test results files found
I’ve just run a build on one of my projects, and the test and coverage results seem to be correctly found:
2019-01-09T14:07:33.9623117Z 14:07:33.663 Fetching code coverage report information from TFS...
2019-01-09T14:07:33.9623193Z 14:07:33.665 Attempting to locate a test results (.trx) file...
2019-01-09T14:07:33.9623430Z 14:07:33.671 Looking for TRX files in: D:\a\1\TestResults, D:\a\1\s\TestResults
2019-01-09T14:07:33.9623520Z 14:07:33.671 The following test results files were found: D:\a\1\s\TestResults\VssAdministrator_fv-az24 2019-01-09 14_07_26.trx
2019-01-09T14:07:33.9623609Z 14:07:33.683 The following code coverage attachments were found from the trx files: D:\a\1\s\TestResults\VssAdministrator_fv-az24 2019-01-09 14_07_26\In\fv-az24\VssAdministrator_fv-az24 2019-01-09 14_06_54.coverage
, and therefore correctly reported in SonarCloud (see the results for my Sample .NET Project).
Can this come from missing or incorrect configuration of one of the tasks?
What version of the vstest task is you using, as you can se in the repo I provided I have been using v2 and the hosted (vs 2015) build image.
If you also look at the Microsoft github issue this is a (recent)known change to newer agents and it seems they don’t want users to use the test result directly
I’ve tried to change the vstest task back to v1 and then it works, but v2 does not work (which I currently “need” in order to solve some flaky tests).
If you work at sonarsource I highly recommend you to visit the github issue at microsoft repository (my first link) since at least some developer over there seems to have a goal of not having any build results accessible to other tasks (except by maybe downloading them from artifact)
Do we have a solution here, I have pressure mounting to remove sonarcloud and use codecov, as sonar codecoverage is showing 0.0%, tried all options, seems a bug in sonar. Codecov just scans and shows the report in dashboard, i ran corbetura and it generates the xml file in the test, and i can see the code cov report, but cannot find this in sonar, any help , else need to go away from sonar…SAD…it failed
Update: we think we’ve identified the root cause of the issue. It isn’t specifically a problem with v2 of the VSTest task (we use VSTest v2 for our internal builds and we haven’t had any problems with coverage being uploaded). If the output for the VSTest v2 step contains the output Updated Run Settings: then we think you will be affected by this issue.
Instead, it depends on which properties you set for the test step. If you change some of the settings from the defaults (e.g. runInParallel, runOnlyImpactedTests, rerunFailedTests …) then the VSTest task will generate a runsettings file that changes the location of the test output files so the analyze step doesn’t find them.
Many thanks @Daniel-Svensson for the links and the pointers to your repo which greatly helped diagnosing the issue. FYI I forked your repo and tweaked the azure-pipelines.yml. Disabling a couple of the more advanced test settings stopped the VSTest task from generating a new runsettings file which meant the coverage data was uploaded successfully. Unfortunately, these are probably exactly the test settings you need to set to handle your flaky tests.
We’re considering a number of options to fix the bug. The one suggested by Microsoft (use the rest APIs to check for the test artefacts and re-download them to the build agent) is exactly what we do for the legacy XAML builds. In practice, it’s quite an ugly solution - the test artefacts need to be processed on the server before they are available via the Azure REST APIs, so if we try to query the REST APIs too soon then we get a response saying there are no artefacts. This gives us a problem - it might be the case that there really are no results, or it might be that there are test artefacts but they haven’t been processed yet. The solution we use for the legacy XAML builds is to poll the TFS server at intervals then timeout after a few retries. Ugly, not very reliable, and it slows down the build unnecessarily if there are in fact no test results. However, we might have to use a similar solution if we can’t find anything better.
Our aim is to release an new version of the extension at the end of this week. We hope it will contain a fix for this issue, or at least a workaround until we can come up with a complete fix.
Update on this thread: the ticket (VSTS-179) just got fixed and deployed (kudos to @duncanp!). Things should be OK by now. Please feel free to ping us on this thread if you are still experiencing problems.
@duncanp I’m using VSO with the latest SonarCloud extensions. I still cannot see the codecoverage in sonarcloud. I use the VsTest task, and see that it writes it output to %drive%\work_temp\TR_%guidlike%. Do i need to alter something to make it work? I tried various things but can’t get the tests & codecoverage in results in sonarcloud
Also doing static code analysis with Azure DevOps. I’ve been trying to experiment with different configurations but coverage is always 0% in sonar cloud
some logs:
SonarScanner for MSBuild 4.6.2
Using the .NET Core version of the Scanner for MSBuild
Pre-processing started.
…
sonar.organization=leasecrunch
sonar.host.url=https://sonarcloud.io/
sonar.projectKey=LeaseCrunch.Core
sonar.projectName=LeaseCrunch.Core
sonar.projectVersion=2.8.0.7
sonar.branch.name=develop
sonar.cs.vscoveragexml.reportsPaths=/home/vsts/work/_temp/**/*.trx
sonar.visualstudio.enable=false
…
22:09:19.338 INFO: Sensor C# Tests Coverage Report Import [csharp]
22:09:19.355 INFO: Parsing the Visual Studio coverage XML report /home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_39.trx
22:09:19.360 WARN: Could not import coverage report ‘/home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_39.trx’
22:09:19.360 INFO: Parsing the Visual Studio coverage XML report /home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_30.trx
22:09:19.361 WARN: Could not import coverage report ‘/home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_30.trx’
22:09:19.361 INFO: Parsing the Visual Studio coverage XML report /home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_35.trx
22:09:19.362 WARN: Could not import coverage report ‘/home/vsts/work/_temp/_fv-az606_2019-07-26_22_08_35.trx’
22:09:19.362 INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=24ms
I suggest you to open a new thread with this issue, run the analysis with either System.debug variable at true (or sonar.verbose at true) and post the result, sounds like a parsing issue.