1.6.0 SonarCloud VSTS plugin update causes files not to be processed

SonarQube Cloud
1

Since the morning of Feburary 27th our previously working typescript and HTML file analysis is no longer working.

Here is a section of our logs in one of the affected areas.

Feb 26th:

INFO: ------------- Run sensors on module Web
INFO: Sensor SonarCSS Metrics [cssfamily]
INFO: Sensor SonarCSS Metrics [cssfamily] (done) | time=368ms
INFO: Sensor SonarCSS Rules [cssfamily]
INFO: Sensor SonarCSS Rules [cssfamily] (done) | time=5484ms
INFO: Sensor PL/SQL Sensor [plsql]
INFO: 4 source files to be analyzed
WARNING: WARN: 
WARNING: WARN: Unable to fully parse: D:/_work/2/s/src/Web/Service/ConfirmationsTestService.ClearInbound.sql
WARNING: WARN: Parse error starting from line 1
WARNING: WARN: 
WARNING: WARN: 
WARNING: WARN: Unable to fully parse: D:/_work/2/s/src/Web/Service/ConfirmationsTestService.ClearTrades.sql
WARNING: WARN: Parse error starting from line 1
WARNING: WARN: 
INFO: 4/4 source files have been analyzed
INFO: Sensor PL/SQL Sensor [plsql] (done) | time=26ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor JavaXmlSensor [java]
INFO: 15 source files to be analyzed
INFO: Sensor JavaXmlSensor [java] (done) | time=127ms
INFO: Sensor HTML [web]
INFO: 15/15 source files have been analyzed
INFO: Sensor HTML [web] (done) | time=924ms
INFO: Sensor XML Sensor [xml]
INFO: 15 source files to be analyzed
INFO: Sensor XML Sensor [xml] (done) | time=139ms
INFO: Sensor SonarTS [typescript]
INFO: 15/15 source files have been analyzed
INFO: Analyzing 8 typescript file(s) with the following configuration file D:\_work\2\s\src\Web\App_Test\Mock\tsconfig.json
INFO: 8 files analyzed out of 8
INFO: Analyzing 457 typescript file(s) with the following configuration file D:\_work\2\s\src\Web\App\Script\tsconfig.json
INFO: 100 files analyzed out of 457. Current file: D:\_work\2\s\src\Web\App\Script\Core\Models\Dialog\Dialog.ts
INFO: 256 files analyzed out of 457. Current file: D:\_work\2\s\src\Web\App\Script\Directives\ExpressionBuilder\ExpressionBuilderController.ts
INFO: 420 files analyzed out of 457. Current file: D:\_work\2\s\src\Web\App\Script\Test\Controllers\ComponentsTestController.ts
INFO: 457 files analyzed out of 457
INFO: Analyzing 46 typescript file(s) with the following configuration file D:\_work\2\s\src\Web\App_Test\tsconfig.json
INFO: 46 files analyzed out of 46
INFO: Sensor SonarTS [typescript] (done) | time=43043ms
INFO: Sensor SonarTS Coverage [typescript]
INFO: Analysing [D:\_work\2\s\src\Web\App_Test\Coverage\lcov.info]
WARNING: WARN: Could not resolve 34 file paths in [D:\_work\2\s\src\Web\App_Test\Coverage\lcov.info], first unresolved path: App\Script\helpers\typehelper.ts
INFO: Sensor SonarTS Coverage [typescript] (done) | time=139ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=0ms

Feb 27th:

INFO: ------------- Run sensors on module Web
INFO: 1/1 source files have been analyzed
INFO: Sensor SonarCSS Metrics [cssfamily]
INFO: Sensor SonarCSS Metrics [cssfamily] (done) | time=893ms
INFO: Sensor SonarCSS Rules [cssfamily]
INFO: Sensor SonarCSS Rules [cssfamily] (done) | time=43508ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor JavaXmlSensor [java]
INFO: 15 source files to be analyzed
INFO: Sensor JavaXmlSensor [java] (done) | time=408ms
INFO: Sensor XML Sensor [xml]
INFO: 15/15 source files have been analyzed
INFO: Sensor XML Sensor [xml] (done) | time=0ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=15ms

These two builds were done on the same branch, one at feb 26 at 6:13 pm, and the other at feb 27 at 7:27 am.

There were no changes to the project configuration or sonarcloud configuration during this time. The analysis has not worked since.

2

Could you show full analysis log?
Looks like scanner does not see any HTML or TS files in your project.

3

Feb26log.txt (58.8 KB)

Feb27log.txt (57.2 KB)

4

In fact, looking at our main project, it’s no longer detecting any files in our /src/Web folder at all any more.

5

I had previously configured a Test Inclusion path for src/Web/App_Test and now this is the only folder showing up here, despite there being a .csproj that points at this folder.

testInclusion
testInclusion.png1205×32 1.13 KB

6

@Lena have you had a chance to have a look at why this might be happening yet?

7

Sorry, me again. I’ve done a bit more analysis on our side.

It seems like the version upgrade of the SonarCloud extension from 1.5.1 to 1.6.0 is what broke our process. More specifically we expect that it was the upgrade of the MSBuild scanner from version 4.5.0.1761 to 4.6.0.1930.

We can actually see the new versions being downloaded in the build that broke our process.

##[section]Starting: Initialize job
Current agent version: '2.147.1'
Prepare build directory.
Set build variables.
Download all required tasks.
Downloading task: SonarCloudPrepare
Downloading task: SonarCloudAnalyze
Downloading task: SonarCloudPublish
Start tracking orphan processes.
##[section]Finishing: Initialize job
8

VSTS extension was indeed updated from 1.5 to 1.6 version, however this should be transparent if no changes were done to the project or analysis configuration. To debug the problem could you attach DEBUG log of analysis (by passing sonar.verbose=true to Prepare analysis on SonarQube step and tell me which files, that you expect to be analyzed do not show up.

9

Okay, I shall run that analysis, and get back to you. Is there any way I can send the logs to you over a non-public forum?

11

I’ve sent those logs through now. Please let me know if you need anything else.

12

This exact same problem happened to us, it started marking all our files as test files rather than actual files to scan, I think it was thinking our project was a test project.

I had to put
<SonarQubeTestProject>false</SonarQubeTestProject>
inside our csproj file for it to start scanning normally again…

1 Like
13

Thanks, I’ll give that a go and see what happens.

16

FYI we’ve created https://github.com/SonarSource/sonar-scanner-msbuild/issues/764 to track this issue.

17

@StephenLarkin I’ve had a look at the web.csproj you supplied to @budzow, which was one of the projects that changed from being classified as a product project to a test project.

The project contains the following setting:

<ItemGroup>
  <Service Include="{82A7F48D-3B50-4B1E-B82E-3ADA8210C358}" />
</ItemGroup>

That service tag is used by the Test Explorer in older versions of VS to identify certain types of test project, according to this StackOverflow answer. Scanner ticket #651 included this change so it recognised projects with that service tag as test projects.

That explains why the analysis of your projects changed between v4.5 and v4.6 of the Scanner for MSBuild, and it seems that treating projects with this tag as test projects is the correct thing to do as it is the approach used by VS (strictly speaking, it was the approach used by VS; newer versions of VS/MSBuild use different mechanisms, so use of the service tag is a legacy behaviour).

However, it doesn’t explain why your product project contains the service tag since it isn’t something that would normally be added manually. The only other options I can think of is that the project once did contain tests that have since been removed, or that VS incorrectly applied the service tag to the project.

Either way, the service tag can be safely removed from the product projects, after which the scanner should correctly identify them as product projects.

1 Like
19

Thanks very much. It is likely that at some stage some tests were added in the past that were then removed again.

Thank you very much for your investigation into this issue, we will update our csproj files accordingly.