I have a build pipeline where i have integrtaed sonar prepare analysis ,Run and publish analysis tasks configured.
I see the unit tests are getting executed and also there is a coverage report which i download and see in the VS 2017 but the build fails showing code coverage is less than< 80 as that is org level check/setting.
Prepare Analysis has below setting
sonar.cs.vstest.reportsPaths=(Agent.TempDirectory)\**\*.trx
sonar.cs.vscoveragexml.reportsPaths=(Agent.TempDirectory)***.coveragexml
Have attached the error log which i get in the Run analysis step.I see the cooveragexml is also getting generated but in logs it is showing skipping Test files see small snippet below.
Parsing the Visual Studio coverage XML report D:\a\_temp\TestResults\2a78726f-c71e-4e9a-baa0-769dd16b026b\VssAdministrator_fv-az121-164_2021-04-09.12_49_21.coveragexml
2021-04-09T12:52:38.8060448Z 12:52:38.770 INFO: Adding this code coverage report to the cache for later reuse: D:\a_temp\TestResults\2a78726f-c71e-4e9a-baa0-769dd16b026b\VssAdministrator_fv-az121-164_2021-04-09.12_49_21.coveragexml
2021-04-09T12:52:38.8061243Z 12:52:38.771 DEBUG: Analyzing coverage after aggregate found â6â coverage files.
2021-04-09T12:52:38.8061937Z 12:52:38.771 DEBUG: Skipping âD:\a\1\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\UserDataManagerUnitTest.csâ as it is a test file.
2021-04-09T12:52:38.8062744Z 12:52:38.772 DEBUG: Skipping âD:\a\1\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalBusinessManagerUnitTest.csâ as it is a test file.
2021-04-09T12:52:38.8063841Z 12:52:38.772 DEBUG: Skipping âD:\a\1\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalIntegrationBusinessManagerUnitTest.csâ as it is a test file.
2021-04-09T12:52:38.8064692Z 12:52:38.772 DEBUG: Skipping âD:\a\1\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalIntegrationDataManagerUnitTest.csâ as it is a test file.
2021-04-09T12:52:38.8065482Z 12:52:38.772 DEBUG: Skipping âD:\a\1\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalDataManagerUnitTest.csâ as it is a test file.
2021-04-09T12:52:38.8066285Z 12:52:38.772 DEBUG: Skipping âD:\a\1\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\UserBusinessManagerUnitTest.csâ as it is a test file.
2021-04-09T12:52:38.8067387Z 12:52:38.772 DEBUG: The total number of file count statistics is â6â.
2021-04-09T12:52:38.8079418Z 12:52:38.773 INFO: Coverage Report Statistics: 6 files, 0 main files, 0 main files with coverage, 6 test files, 0 project excluded files, 0 other language files.
2021-04-09T12:52:38.8556733Z 12:52:38.773 WARN: The Code Coverage report doesnât contain any coverage data for the included files. For troubleshooting help, please visit our community forum
sonarlatest.zip (430.4 KB) sonarlatest.zip (430.4 KB)
Hi @anoop52172,
in order to be able to help we would need more information. Could you please run the analysis in verbose mode (you can enable verbose mode by adding /d:sonar.verbose=true parameter to the begin step) and send to us the output of the end step. (You can redirect the output to a file, zip it and attach it to this thread.)
Thanks,
Costin
in the coveragexml file I am able to see below information,it identifies the test files but not gives coverage for the methods for which the tests are written
function id=â8272â token=â0x6000001â name=âGetPortalsWithRelatedUserWithPaginationSuccessTest()â namespace=âCCH.UnitTest.PotalService.DataManagerâ type_name=âPortalIntegrationDataManagerUnitTestâ block_coverage=â100.00â line_coverage=â100.00â blocks_covered=â18â blocks_not_covered=â0â lines_covered=â27â lines_partially_covered=â0â lines_not_covered=â0â>
[âŚ]
[âŚ]
[âŚ]
[âŚ]
2021-04-22T11:39:22.7328369Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalBusinessManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7329133Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\UserBusinessManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7329823Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalIntegrationDataManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7330925Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalIntegrationBusinessManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7331529Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalDataManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7333356Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\UserDataManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7333743Z 11:39:22.729 DEBUG: The total number of file count statistics is â6â.
2021-04-22T11:39:22.7334142Z 11:39:22.729 INFO: Coverage Report Statistics: 6 files, 0 main files, 0 main files with coverage, 6 test files, 0 project excluded files, 0 other language files.
yes I saw the link but everything is pretty much configured,in my coveragexml below information,it identifies the test files but not gives coverage for the methods for which the tests are written.
function id=â8272â token=â0x6000001â name=âGetPortalsWithRelatedUserWithPaginationSuccessTest()â namespace=âCCH.UnitTest.PotalService.DataManagerâ type_name=âPortalIntegrationDataManagerUnitTestâ block_coverage=â100.00â line_coverage=â100.00â blocks_covered=â18â blocks_not_covered=â0â lines_covered=â27â lines_partially_covered=â0â lines_not_covered=â0â>
[âŚ]
[âŚ]
[âŚ]
[âŚ]
2021-04-22T11:39:22.7328369Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalBusinessManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7329133Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\UserBusinessManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7329823Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalIntegrationDataManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7330925Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalIntegrationBusinessManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7331529Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalDataManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7333356Z 11:39:22.729 DEBUG: Skipping âE:\WKApps\18\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\UserDataManagerUnitTest.csâ as it is a test file.
2021-04-22T11:39:22.7333743Z 11:39:22.729 DEBUG: The total number of file count statistics is â6â.
2021-04-22T11:39:22.7334142Z 11:39:22.729 INFO: Coverage Report Statistics: 6 files, 0 main files, 0 main files with coverage, 6 test files, 0 project excluded files, 0 other language files.
in the coveragexml file I am able to see below information,it identifies the test files but not gives coverage for the methods for which the tests are written
In this case the problem is not on our side since the coverage report is not generated correctly. You need to check if the tool which generates the coverage is called correctly or if there is a bug in the tool.
I am using the azure pipelines ,my files for which the tests are running are getting analyzed ,my tests run is howing up in sonar but somehow coverage for the files is not showing up ,donât know what i am missing here.
Hi @anoop52172, as far as I can tell the coverage file contains information only about the actual test files and not the production code.
Could you please check the contents of the generated coverage file to see if the methods you expect to be covered are included? If they are not in the report we cannot import them.
You could also archive and attach the coverage report so we can take a look.
Thanks,
Costin
Hi Costin ,thanks for your reply coverage.zip (2.6 KB)
attaching the coverage xml.
Also attaching the tests that run and the coverage file that is getting generated. have also attached the screenshot of the pipeline you can see the sonar analysis task and the test soln(highlighted in the pipeline) which has the tests and also the code for which the tests are running.
PFA zip file
SonarWebAPI.zip (155.7 KB)
I also see this in the sonar analysis
Apps_temp\TestResults\3727b65b-0235-4577-a4a5-2cc3a7fa632c\build.tfs_VSTS-BA-62X 2021-05-24 06_13_29.coveragexml
2021-05-24T06:17:09.3300177Z 06:17:09.314 INFO: Adding this code coverage report to the cache for later reuse: E:\WKApps_temp\TestResults\3727b65b-0235-4577-a4a5-2cc3a7fa632c\build.tfs_VSTS-BA-62X 2021-05-24 06_13_29.coveragexml
2021-05-24T06:17:09.3301423Z 06:17:09.314 DEBUG: Analyzing coverage after aggregate found â11â coverage files.
2021-05-24T06:17:09.3303020Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\UserBusinessManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3303775Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\LoginBusinessManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3304429Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalExportBusinessManagerTests.csâ as it is a test file.
2021-05-24T06:17:09.3305085Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalDataManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3305969Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\DocumentDataManagerTests.csâ as it is a test file.
2021-05-24T06:17:09.3306682Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\UserDataManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3307513Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalIntegrationDataManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3308397Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalDocumentExportBusinessManagerTests.csâ as it is a test file.
2021-05-24T06:17:09.3309283Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\DataManager\PortalExportDataManagerTests.csâ as it is a test file.
2021-05-24T06:17:09.3310220Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalIntegrationBusinessManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3311132Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\PortalBusinessManagerUnitTest.csâ as it is a test file.
2021-05-24T06:17:09.3311867Z 06:17:09.314 DEBUG: The total number of file count statistics is â11â.
I have a question ,it is saying skiping as these are test files my code/methods(for which tests are written)is written in diff project but in the same soln why is it not able to determine the tests are running for those and give me the coverage
Hi @anoop52172
thanks for the attached files. Iâve took a look in the coverage xml and, as you can see there is only one module cch.unittest.potalservice.dll analyzed and this one contains the test code. There are no details there about the production code so we donât have what to import.
The log lines you pasted similar with:
2021-05-24T06:17:09.3303020Z 06:17:09.314 DEBUG: Skipping âE:\WKApps\45\s\Source\Source\Tests\CCH.PotalServiceUnitTests\BusinessManager\UserBusinessManagerUnitTest.csâ as it is a test file.
are perfectly normal since we donât import coverage for test files.
How do you call the coverage tool? I can see in the picture that there are two build steps one for the main code and one for the tests. I think this is the reason why visual studio coverage tool is not computing the coverage for the main code files.
Hi Costin, We have multiple solution and multiple projects ,so in screenshot1 I have a test solution which has Core project (production code Core project etc ) and test project also.
Similarly in second screenshot 2 you can see I have portal services soln which also has core project along with other projects.
In my pipeline I have tried building my services soln as i wan all code to be analyzed from the build task and then i am building the test project before running it.
Should i just build my test soln screenshot 1 as the production code will get analyzed in the build task/step??
As you can see from VS i am able to see coverage properly when I analyze code coverage from test solution. As shown the core project shows uncovered lines but the one in azure (generated from build pipeline shows only test file data).
Hi @anoop52172,
I have to highlight that this issue is not on our side (none of the production code metrics are included in the coverage file you attached). I understand that you can see the coverage in visual studio but please check the file yourself (coverage xml generated by visual studio coverage) to see that the metrics for the production code are not there.
You have to check the docs (Microsoft) to see how to call the build to include in the coverage results (xml file) the metrics for the production code.
1 Like
