Error message from "Prepare analysis on SonarQube" Azure DevOps task v5.6.0

“Prepare analysis on SonarQube” Azure DevOps task v5.6.0
SonarScanner for MSBuild 5.7.1

Pipeline which was running successfullly on 2022-07-04,
started failing on 2022-07-05 with message:

(…).sonarqube\bin\targets\SonarQube.Integration.targets(241,5): error : Failed to parse properties from the environment variable ‘SONARQUBE_SCANNER_PARAMS’ because ‘Could not load file or assembly ‘Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed’. Could not find or load a specific file. (Exception from HRESULT: 0x80131621)’. [(…).csproj] [(…)\MasterBuild.proj]

looks like it may be related with

sonar-scanner-msbuild/EnvScannerPropertiesProvider.cs at 05499245d82522b5543db07355cc20b10631335a · SonarSource/sonar-scanner-msbuild · GitHub

commited:

1 Like

Hello @pawel.grzywacz

Indeed we have deployed a new version of the Scanner for .NET and indeed we have fixed a bug that was ignoring the environment variables.

Could you please give us the verbose logs (pass sonar.verbose to the PrepareAnalysis step) of the build?

Does the project you analyze reference Newtonsoft.Json , too?

1 Like

Hi Andrei, thank you for picking my issue up instantly.

Here goes the verbose log from the “Prepare*” task.
build.424591.log.verbose.txt (8.7 KB)

Answering your questiion, yes we are referencing Newtonsoft.Json a lot, the version we used in the sources for this pipeline was 11.0.2.

here goes the exact error from our VisualStudio Build pipeline step:

##[error]D:\B\A1_work\3523.sonarqube\bin\targets\SonarQube.Integration.targets(241,5): Error : Failed to parse properties from the environment variable ‘SONARQUBE_SCANNER_PARAMS’ because ‘Could not load file or assembly ‘Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed’. Could not find or load a specific file. (Exception from HRESULT: 0x80131621)’. [D:\B\A1_work\3523\s\Foundation\APIs\Common\Common.csproj]
D:\B\A1_work\3523.sonarqube\bin\targets\SonarQube.Integration.targets(241,5): error : Failed to parse properties from the environment variable ‘SONARQUBE_SCANNER_PARAMS’ because ‘Could not load file or assembly ‘Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed’. Could not find or load a specific file. (Exception from HRESULT: 0x80131621)’. [D:\B\A1_work\3523\s\Foundation\APIs\Common\Common.csproj] [D:\B\A1_work\3523\s\Foundation\Build\Tasks\MasterBuild.proj]

Just in order to reanimate our pipelines, we disabled the “Prepare analysis on SonarQube” step temporarily.

Regards!

Hi @pawel.grzywacz

Thanks for the logs of the Begin step.

Sorry, when I meant build I meant all the steps, also the MSBuild verbose logs (this is the step where it’s failing, from what I understand). And for that you’ll need to pass /v:d.

Hi, the log contains >500K lines, I’ve cut what I thought to be of a least importance here out.
build.424664.log.verbose.cut.txt (40.6 KB)

Thanks, however this doesn’t help me understand why the task is not picking up the correct Newtonsoft version.

In PR # 1246 I made a necessary change to copy Newtonsoft.Json.dll (we reference v 13) to the .sonarqube\bin folder which is present in the base directory of the analysis. That folder has the DLLs with the custom Sonar tasks that get executed during the build to prepare the analysis that happens during the compilation.

This build fails because the compiler doesn’t manage to reference that version of Newtonsoft (or doesn’t reference it at all for any reason).

So I need to look at the csc.exe command with all its long list of parameters (it will show all the DLLs that get passed).

For reference, I see from the logs that you use Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core - what .NET Core version are you using? If it’s .NET Core 2.1, I believe it might fail because that one references an older version of Newtonsoft (although in our tests with dotnet core 2.1 it doesn’t, there may be something stranger happening).

Well, its even .Net Framework.

The projects inside Foundation\APIs\APIs.sln
have net461

Thanks!

Can you help me please with the csc.exe command from the logs? Or share the full log - I’m not afraid of big log files, and having access to the full logs enables me to do my search.

@pawel.grzywacz you can share the full logs privately here: https://community.sonarsource.com/t/re-error-message-from-prepare-analysis-on-sonarqube-azure-devops-task-v5-6-0/67509

Hi Andrei,

I’ll share the logs first thing in the morning.
Apologies for the delay.

Regards.

Hi @Andrei_Epure

I am having the same exact problem. An analysis that was working correctly until July 1st, started to break today (July 5th). Same exact error:

##[error]D:\a\1\.sonarqube\bin\targets\SonarQube.Integration.targets(241,5): Error : Failed to parse properties from the environment variable 'SONARQUBE_SCANNER_PARAMS' because 'Could not load file or assembly 'Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621)'.
D:\a\1\.sonarqube\bin\targets\SonarQube.Integration.targets(241,5): error : Failed to parse properties from the environment variable 'SONARQUBE_SCANNER_PARAMS' because 'Could not load file or assembly 'Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621)'. [D:\a\1\s\myWorld\myWorld.csproj]

Our project wasn’t referencing Neewtonsoft.JSON before and we added the reference to see if it fixes but we had no luck, same error.

<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />

Are there any steps we can follow to have the pipeline and tasks working again?

BTW, I am using SonarCloud and the build agent is Azure Pipelines

Hello @caraujo84

No, we reference newtonsoft and provide the dependency explicitly and wherever it is necessary.

Could you please share the verbose logs of your msbuild / dotnet build, please (/v:d)?

Hello @pawel.grzywacz and @caraujo84

We’ve published new versions of the Azure DevOps extensions, reverting the Scanner for .NET (S4NET) to the previous version (5.6.0) - this should unblock you.

I’ve opened Log warning instead of error when not parsing environment variables to avoid hard failure when Newtonsoft does not get resolved · Issue #1281 · SonarSource/sonar-scanner-msbuild · GitHub to fix / mitigate this issue on the S4NET side.

In any case, the root cause is the Newtonsoft DLL not being resolved and we still do not understand this behaviour in your build environment.

@pawel.grzywacz , @caraujo84 Are you using Microsoft hosted agents or self-hosted agents for your builds?

@pawel.grzywacz The logs say Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core, so I would assume there is some .NET Core installation on the box. In any case, the .NET Core version of MSBuild is the one doing the build (even if it’s a .NET Framework project). Is it the case?

Thanks so much Andrei.

As per the first question - we are using our own agents.

As per the second - your assumption is adequate, there is .Net Core 2.1 installed on agents running this particular application builds.

Best regards, Paweł

Thanks, @Andrei_Epure . Reverting back to 5.6.0 did the trick, things are working as usual.

Are you using Microsoft hosted agents or self-hosted agents for your builds?

We are using Microsoft-hosted agents and the solution is using .Net Core 2.2

Thanks for confirming this assumption @caraujo84 and @pawel.grzywacz . We’ve done tests with machines with .NET Core 2.1, however we might have missed something.

FYI .NET Core 2.1 is not supported by Microsoft since August 2021. We have announced stopping support for it :

However we’ve not put that in practice yet, for now it’s deprecated. In the next months we will stop supporting it, though, so I encourage you to use .NET Core Runtime versions supported by Microsoft: .NET and .NET Core official support policy . It’s fine if your projects are still targeting .NET Core 2.1, as long as the runtime is supported (the SDK).

The same for .NET Framework, you should use at least .NET Framework 4.6.2 .NET Framework official support policy

@pawel.grzywacz , @caraujo84 - I will continue on private threads to better understand your environments and why this failure occurs, because we still don’t have a reproducer for the problems you encounter.