Yesterday, Nov 13, two of our teams started seeing a sudden drop in their coverage and lines to cover. The first team had:
12 Nov: 149K lines to Cover, 11K Covered lines, 139K uncovered → 7.2% code coverage
13 Nov: 139K lines to Cover, 6.5K Covered lines, 132K uncovered lines → 4.7% code coverage
This makes us wonder why these are now excluded? According to the teams, they have not made any changes related to these items. So did something change on SonarCloud? E.g. exclusions that did not work before that work now maybe?
PS: We did see that in the processing of “Run analysis” a lot less files were analyzed than before. But right now we are in a blank whether it is the Microsoft hosted agent or SonarCloud.
Hi @mickaelcaro,
We use Azure DevOps hosted build agent, latest version of the scanner task. Technologies used: C#, JavaScript, XML, HTML, Typescript and CSS (typical Microsoft Intranet applications) for both applications. T-SQL for the database.
From what i saw (I looked at the “CreditHub” log files only for now"), there have been an increased number of files indexed by our sensors i.e 1912 files on the 12th and 1949 on the 13th, so to me it’s normal at that only point that the coverage decrease if those new files were not covered by Unit Test.
We can see also that, given the metric of the log itself, it says on the 12th that :
Coverage Report Statistics: 1305 files, 696 main files, 696 main files with coverage, 609 test files, 0 project excluded files, 0 other language files.
Whereas on the 13th :
Coverage Report Statistics: 979 files, 362 main files, 362 main files with coverage, 617 test files, 0 project excluded files, 0 other language files.
So indeed something has changed here.
As a further reading, can you please provide me, if you have access to, the scanner context that are available in the Background tasks section? Ideally again one of the 12th and the 13th.
Hi,
Indeed, I see the file difference. Weird thing is that both teams did not make any change in their solution/build. Requested scanner files attached. Who determines the nr of files and the coverage? Is that the MS Build or Sonar scanner? As could it also be that previous file exclusions were for instance not taken into account, and they are taken into account now? Or a miscalculation somewhere that is now fixed?
Hi Mickaël,
I downloaded the code coverage reports of both 12 and 13 november of the main branch of both applications, and both went from a 18-19Mb file on the 12th to a 14-15Mb file on the 13th.
So roughly 4Mb less, and also slightly different coverage (but only 1% on eCS, not the 3% in SonarCloud).
Is it the SonarCloud extension telling MS Build which parts to cover for the code analysis, or is this purely Microsoft side?
SonarCloud and its subsequent analyzers are only based on what we feed them in terms of code coverage (trx / coveragexml files if VSTest is used), then a match is made between coverage metrics and files that have been analyzed.out
There could be a slight difference between what is uploaded as a summary in the build against the coverage shown on SonarCloud simply because some files might not have been analyzed or are not supported whether the coverage shown in the summary is kinda “raw”
If there have been a drop on the coverage file itself, then there’s not that much we can do unfortunately.