MSBuild mode: How to scan c#, csproj and nuget.config files in same scan?

Andre_Hansen · May 3, 2024, 2:32pm

In short
I have written custom XPATH xml rules to detect problems with the nuget.config and csproj, i have confirmed the rules work when not using scannerMode MSBuild. However, trying combine MSBuild to detect c# problems with xml rules seems damn freaking hard…

The fact that i cant easily setup a scan of a folder structure containing c# and other non-c# files, but are instead forced into this idea that when your scanning a .net project you can only scan the content of the csproj-projects folders is annoying at best.

I have a folder structure like this

Source
   InternalApi
      nuget.config         <- Is scanned, but only added it for testing
      InternalApi.csproj   <- Not scanned
   nuget.config            <- Not scanned

We are using Azure DevOps pipelines

- task: SonarQubePrepare@5
  displayName: Sonarqube prepare
  inputs:
    SonarQube: $(sonarqube_service_connection)
    scannerMode: 'MSBuild'
    projectKey: $(sonarqube_projectKey)
    projectName: $(sonarqube_projectName)
    cliSources: '$(Build.SourcesDirectory)'
    extraProperties: |
      sonar.cs.vscoveragexml.reportsPaths=$(coverage_file_path)
      sonar.log.level=DEBUG
      sonar.projectBaseDir=$(Build.SourcesDirectory)
- task: DotNetCoreCLI@2
  displayName: 'dotnet build'
  inputs:
    command: build
    projects: '$(path_to_solution)'
    arguments: --configuration $(BuildConfiguration) --no-restore
- task: SonarQubeAnalyze@5
  displayName: Sonarqube analyze

Stripped InternalApi.csproj file

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <SQAdditionalAnalysisFileItemTypes>config;csproj</SQAdditionalAnalysisFileItemTypes>
  </PropertyGroup>
</Project>

I have tried to run two separate analyze jobs, but the last analyze overwrite the first. Which in effect deletes comments on the Azure DevOps PR.

I struggle with

Detect the csproj file within the project
Detect the nuget.config file outside the project structure
Run two separate scans and combine the result

Extra info

which versions are you using: 9.9.1.69595
how is SonarQube deployed: Helm

Relevant logs from azure devops pipeline step

Starting: Sonarqube prepare
==============================================================================
Task         : Prepare Analysis Configuration
Description  : Prepare SonarQube analysis configuration
Version      : 5.19.1
Author       : sonarsource
Help         : Version: 5.19.1. [More Information](http://redirect.sonarsource.com/doc/install-configure-scanner-tfs-ts.html)
==============================================================================
/usr/bin/dotnet /agent/_work/_tasks/SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157/5.19.1/dotnet-sonar-scanner-msbuild/SonarScanner.MSBuild.dll begin /k:SystemInfo
SonarScanner for MSBuild 5.15
Using the .NET Core version of the Scanner for MSBuild
Pre-processing started.
Preparing working directories...
13:27:43.079  Updating build integration targets...
13:27:43.842  Fetching analysis configuration settings...
13:27:44.203  Provisioning analyzer assemblies for cs...
13:27:44.204  Installing required Roslyn analyzers...
13:27:44.205  Processing plugin: csharp version 8.51.0.59060
13:27:44.788  Processing plugin: vbnet version 8.51.0.59060
13:27:44.84  Processing plugin: securitycsharpfrontend version 9.9.0.19083
13:27:45.024  Provisioning analyzer assemblies for vbnet...
13:27:45.024  Installing required Roslyn analyzers...
13:27:45.024  Processing plugin: csharp version 8.51.0.59060
13:27:45.025  Processing plugin: vbnet version 8.51.0.59060
13:27:45.109  Downloading cache. Project key: SystemInfo, branch: master.
13:27:45.148  Incremental PR analysis: 0 files out of 150 are unchanged.
13:27:45.207  Pre-processing succeeded.
Finishing: Sonarqube prepare

Starting: Sonarqube analyze

==============================================================================

Task : Run Code Analysis

Description : Run scanner and upload the results to the SonarQube server.

14:05:59.779 The exclude flag has been set so the project will not be analyzed. Project file: /agent/_work/12/s/Source/Sec_SystemInfo.Migrations/Sec_SystemInfo.Migrations.csproj
14:05:59.786 Using longest common projects path as a base directory: '/agent/_work/12/s/Source'.
14:05:59.81 File was referenced by the following projects: '/agent/_work/12/s/Source/InternalMessaging.Batch.Tests/InternalMessaging.Batch.Tests.csproj', '/agent/_work/12/s/Source/InternalApi.Tests/InternalApi.Tests.csproj', '/agent/_work/12/s/Source/Logic.Tests/Logic.Tests.csproj'.
14:05:59.821 Dumping content of sonar-project.properties
------------------------------------------------------------------------
sonar.projectKey=SystemInfo
sonar.working.directory=/agent/_work/12/.sonarqube/out/.sonar
sonar.projectBaseDir=/agent/_work/12/s/Source
sonar.pullrequest.cache.basepath=/agent/_work/12/s

0E72F32E-300D-4A69-839E-6FEED8453AB0.sonar.projectKey=SystemInfo:0E72F32E-300D-4A69-839E-6FEED8453AB0
0E72F32E-300D-4A69-839E-6FEED8453AB0.sonar.projectName=InternalApi
0E72F32E-300D-4A69-839E-6FEED8453AB0.sonar.projectBaseDir=/agent/_work/12/s/Source/InternalApi
0E72F32E-300D-4A69-839E-6FEED8453AB0.sonar.sourceEncoding=utf-8
0E72F32E-300D-4A69-839E-6FEED8453AB0.sonar.sources=\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/ApisController.cs",\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/ApplicationInstancesController.cs",\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/ApplicationsController.cs",\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/ErrorCodesController.cs",\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/LibrariesController.cs",\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/MessageTypesController.cs",\
"/agent/_work/12/s/Source/InternalApi/Controllers/v1/TestController.cs",\
"/agent/_work/12/s/Source/InternalApi/Program.cs",\
"/agent/_work/12/s/Source/InternalApi/Startup.cs",\
"/agent/_work/12/s/Source/InternalApi/obj/Release/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs",\
"/agent/_work/12/s/Source/InternalApi/obj/Release/InternalApi.AssemblyInfo.cs",\
"/agent/_work/12/s/Source/InternalApi/obj/Release/Helsenorge.SystemInfo.InternalApi.MvcApplicationPartsAssemblyInfo.cs",\
"/agent/_work/12/s/Source/InternalApi/nuget.config",\
"/agent/_work/12/s/Source/InternalApi/web.config",\
"/agent/_work/12/s/Source/InternalApi/Config/InternalApi.Config.dev-mot-k8s.vm.json",\
"/agent/_work/12/s/Source/InternalApi/Config/InternalApi.Config.dev.vm.json",\
"/agent/_work/12/s/Source/InternalApi/Config/InternalApi.Config.json",\
"/agent/_work/12/s/Source/InternalApi/Config/InternalApi.Config.vm.json",\
"/agent/_work/12/s/Source/InternalApi/Dockerfile",\
"/agent/_work/12/s/Source/InternalApi/InternalApi.Octopus.nuspec",\
"/agent/_work/12/s/Source/InternalApi/Properties/launchSettings.json",\
"/agent/_work/12/s/Source/InternalApi/obj/Release/apphost"

ganncamp · May 6, 2024, 2:01pm

Hi,

My first question is whether XML is analyzed at all when you use SonarScanner for .NET, or whether it is analyzed but your rules don’t run?

And what is your project type? “For newer SDK-style projects” everything is supposed to be picked up automatically. Otherwise, you may need to adjust your project files.

Also, the current scanner version is 6.2. I don’t think this will impact your problem, but could you upgrade and try again just to eliminate that variable?

Ann

ganncamp · May 6, 2024, 2:47pm

Hi,

As a followup, it’s been pointed out to me that the latest scanner version isn’t yet available through the ADO extension, so nevermind that part.

Ann

Andre_Hansen · May 6, 2024, 3:02pm

Xml was analyzed, and my rules was hitting where expected (which in end resultet in comments to the azure devops pull request)

And what is your project type? “For newer SDK-style projects” everything is supposed to be picked up automatically. Otherwise, you may need to adjust your project files.

I read clues that one had to include the files in the csproj project one by one file. This is not an option. We are an organization with 150+ csproj files/projects, we cant maintain them based on what an external tool should scan. In addition; including the csproj file itself in the csproj file config seems silly.

ganncamp · May 6, 2024, 3:07pm

Hi,

No, wildcards work. That’s still a lot for 150+ csproj files, but better than one-by-one.

Yeah… I get that. And honestly, I don’t think we ever foresaw wanting to run rules on the project files.

Would you mind sharing what you want to check in them?

Thx,
Ann

Andre_Hansen · May 6, 2024, 4:23pm

I can add images of the rules, easier/quicker

ganncamp · May 6, 2024, 4:26pm

Hi,

Thanks for sharing.

In fact, one of our devs routinely gives a talk about avoiding Nuget dependency confusion attacks, so I’m a little surprised we haven’t beaten you to the punch on this. I’ll make sure the devs see this.

Thx!
Ann

Martin_Strecker · May 7, 2024, 2:34pm

Hi @ganncamp and @Andre_Hansen

We got a related request earlier from the community

We created an issue to track the rule request here:

github.com/SonarSource/sonar-dotnet

New Rule Idea: Mark pre-release nuget package as non compliant

opened 01:46PM - 13 Nov 23 UTC

martin-strecker-sonarsource

Type: Rule Idea

[Community request](https://community.sonarsource.com/t/rule-for-avoiding-pre-re…lease-packages-in-c/103876) ### Reasoning Adding [pre-release](https://learn.microsoft.com/en-us/nuget/concepts/package-versioning#pre-release-versions) package dependency to a project can cause maintainability and security issues. There is a high risk of changing or dropping APIs before a new stable version is released. ### Description Pre-release is a Nuget and not an assembly concept. There are two options for implementing this (further investigation is needed). #### Use the [AssemblyInformationalVersionAttribute](https://learn.microsoft.com/en-us/dotnet/api/system.reflection.assemblyinformationalversionattribute?view=net-7.0) Some packages like [Fody](https://nuget.info/packages/Fody/6.6.5-beta2) or [EF core](https://nuget.info/packages/Microsoft.EntityFrameworkCore/8.0.0-rc.2.23480.1) apply the Nuget version also to the assemblies via the AssemblyInformationalVersionAttribute. We could use [Compilation.References](https://learn.microsoft.com/en-us/dotnet/api/microsoft.codeanalysis.compilation.references?view=roslyn-dotnet-4.7.0) to find the AssemblyInformationalVersionAttribute via the metadata reader of the module of the PortableExecutableReference of external DLLs. Pros: * Would work in the IDE * No "additional files" magic needed Cons: * We would warn pre "DLL" which is not the same as per Nuget package (A single package can contain multiple references, see e.g. [PDFsharp](https://nuget.info/packages/PDFsharp/6.0.0-preview-4)) * The message would be about some dll, which the user needs to map to the Nuget package. We need proper RSpec to help the user do so. * We rely on the AssemblyInformationalVersionAttribute which is not present in all Nuget packages (e.g. [Syncfusion.Xamarin.Pdf](https://nuget.info/packages/Syncfusion.Xamarin.Pdf/17.3.0.9-beta)) or does not include the "pre-release" keywords (e.g. [GraphQL](https://nuget.info/packages/GraphQL/3.0.0-preview-1719)) * Where do we report the issue? * Transitive dependencies would show up as well (packages depending on other packages or references caused by packages of a dependent project) #### Instrument msbuild in the scanner to add the csproj as an additional file This would allow us to access the csproj (and reporting there should also just work). Pros: * We would get the version from the `<PackageReference Version>` attribute, which is the only reliable source of truth * No problem with transitive dependencies and multiple DLLs per package * We can report on the package reference in the csproj * We can create other rules for csproj Cons: * We would rely on changes in the scanner (We need to add the csproj to the additional files, which would need changes in our scanner target file) * This will likely not work in the IDE (maybe connected mode would work)

We also did a short investigation on how to implement such a rule in the .Net analyzer back then. We only checked how such issues are reported in the IDE but not on the SonarQube/SonarCloud servers. We should consider adding *csproj, *.config, *.sln, and similar files by default. Without such files, the code view on the server is missing important artifacts anyway.

The two rules shown by @Andre_Hansen are interesting. Unfortunately, they are very noisy with a high false positive rate. If we add such rules, they will have to be disabled by default, and users will need to opt in explicitly.

Topic		Replies	Views
Is there a possibility to scan the contents of a .csproj file for certain tags with a custom rule SonarQube Server / Community Build xml , sonarqube	2	1645	October 21, 2020
C# project not analyzing SonarQube Server / Community Build csharp , sonarqube , msbuild	1	743	July 24, 2019
MSBuild Scanner issue including external files SonarQube Server / Community Build scanner , dotnet , azuredevops-server	3	1063	February 24, 2022
C# files not scanning SonarQube Server / Community Build csharp , sonarqube , msbuild	5	2721	June 17, 2021
Scanner only scan one directory SonarQube Server / Community Build scanner	5	4227	August 19, 2020

MSBuild mode: How to scan c#, csproj and nuget.config files in same scan?

Related topics