SonarQube: Illegal characters in path

msedi · September 6, 2023, 2:30pm

I’m using SonarQube for code analysis in my azure build pipelines. During my last check-in I get an error message: “Illegal characters in path” without any hint about the location of the problem:

    ##[error]Unhandled Exception:
    ##[error]System.ArgumentException: Illegal characters in path.
       at System.Security.Permissions.FileIOPermission.EmulateFileIOPermissionChecks(String fullPath)
       at System.Security.Permissions.FileIOPermission.QuickDemand(FileIOPermissionAccess access, String fullPath, Boolean checkForDuplicates, Boolean needFullPath)
       at System.IO.FileInfo.Init(String fileName, Boolean checkHost)
       at SonarScanner.MSBuild.Common.ProjectInfoExtensions.<>c.<GetAllAnalysisFiles>b__8_0(String path)
       at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
       at SonarScanner.MSBuild.Shim.PropertiesFileGenerator.ToProjectData(IGrouping`2 projectsGroupedByGuid)
       at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
       at SonarScanner.MSBuild.Shim.PropertiesFileGenerator.TryWriteProperties(PropertiesWriter writer, IEnumerable`1& allProjects)
       at SonarScanner.MSBuild.Shim.PropertiesFileGenerator.GenerateFile()
       at SonarScanner.MSBuild.PostProcessor.PostProcessor.GenerateAndValidatePropertiesFile(AnalysisConfig config)
       at SonarScanner.MSBuild.PostProcessor.PostProcessor.Execute(String[] args, AnalysisConfig config, IBuildSettings settings)
       at SonarScanner.MSBuild.BootstrapperClass.PostProcess()
       at SonarScanner.MSBuild.BootstrapperClass.<Execute>d__8.MoveNext()
    ---

I have already elevated the verbosity to find the problem:

 inputs:
    SonarQube: 'SonarQube'
    scannerMode: 'MSBuild'
    projectKey: 'CT-XXXX'
    extraProperties: |
      sonar.verbose=true
      sonar.log.level=TRACE
      sonar.log.level.app=TRACE
      sonar.log.level.web=TRACE
      sonar.log.level.ce=TRACE
      sonar.log.level.es=TRACE

But didn’t get more information and although I have set it to TRACE it is still:

sonar.verbose=true was specified - setting the log verbosity to 'Debug'

Is there any better switch that prints out the location of the problem?

ALM used: Azure DevOps
CI system used: Azure DevOps
Scanner command used when applicable (private details masked)
Languages of the repository: C#, C++

Colin · September 7, 2023, 4:55pm

Hey there.

Thanks for the report.

Can you upload a larger range of logs? It’s difficult to know where the issue is coming from right now.

msedi · September 7, 2023, 7:06pm

Sure. This is what I have

The pipeline looks like this:

The prepare step is this

- task: SonarQubePrepare@5
  inputs:
    SonarQube: 'IPIPE SonarQube'
    scannerMode: 'MSBuild'
    projectKey: 'CT-XXXX'
    extraProperties: |
      sonar.c.file.suffixes=-
      sonar.cpp.file.suffixes=-
      sonar.objc.file.suffixes=-
      sonar.exclusions=**/*.xml,**/*.xaml,**/*.yml,**/*.csproj,**/*.sln,**/*.slnf,**/*.cu,**/*.ptx,**/*.g.cs,**/*.dcm,**/*.png,**/*.ipv,**/*.ipp,**/*.ipx,**/*.IRSTBL,**/*.java,**/*.jar,**/*.html,**/*.css,**/*.js,**/*.py,**/*.cshtml,Tables/,Tools/,tools.p/,pipelines/,documentation/

The Analyze Step is this

steps:
- task: SonarQubeAnalyze@5
- task: SonarQubePublish@5
  inputs:
    pollingTimeoutSec: '300'

which then shows:

Starting: SonarQubeAnalyze
==============================================================================
Task         : Run Code Analysis
Description  : Run scanner and upload the results to the SonarQube server.
Version      : 5.15.0
Author       : sonarsource
Help         : Version: 5.15.0. This task is not needed for Maven and Gradle projects since the scanner should be run as part of the build.

[More Information](http://redirect.sonarsource.com/doc/install-configure-scanner-tfs-ts.html)
==============================================================================
F:\agent\_work\_tasks\SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157\5.15.0\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe end
SonarScanner for MSBuild 5.13
Using the .NET Framework version of the Scanner for MSBuild
Post-processing started.
16:14:24.982  16:14:24.904  Uninstalling target: F:\agent\_work\2\.sonarqube\bin\targets\SonarQube.Integration.targets
16:14:24.982  16:14:24.982  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
16:14:24.982  Loading the SonarQube analysis config from F:\agent\_work\2\.sonarqube\conf\SonarQubeAnalysisConfig.xml
16:14:24.982  TeamBuild environment detected
16:14:24.982  Analysis base directory: F:\agent\_work\2\.sonarqube
Build directory: F:\agent\_work\2
Bin directory: F:\agent\_work\2\.sonarqube\bin
Config directory: F:\agent\_work\2\.sonarqube\conf
Output directory: F:\agent\_work\2\.sonarqube\out
Config file: F:\agent\_work\2\.sonarqube\conf\SonarQubeAnalysisConfig.xml
16:14:24.997  Generating SonarQube project properties file to F:\agent\_work\2\.sonarqube\out\sonar-project.properties
16:14:25.81  Setting analysis property: sonar.visualstudio.enable=false
16:14:25.825  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\0\Issues.json
16:14:25.825  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\1\Issues.json
16:14:25.841  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\10\Issues.json
16:14:25.841  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\100\Issues.json
16:14:25.841  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\101\Issues.json
16:14:25.857  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\102\Issues.json

showing many messages of the “valid json” type

until here:

   at SonarScanner.MSBuild.Program.<Main>(String[] args)
System.ArgumentException: Illegal characters in path.
   at System.Security.Permissions.FileIOPermission.EmulateFileIOPermissionChecks(String fullPath)
   at System.Security.Permissions.FileIOPermission.QuickDemand(FileIOPermissionAccess access, String fullPath, Boolean checkForDuplicates, Boolean needFullPath)
   at System.IO.FileInfo.Init(String fileName, Boolean checkHost)
   at SonarScanner.MSBuild.Common.ProjectInfoExtensions.<>c.<GetAllAnalysisFiles>b__8_0(String path)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at SonarScanner.MSBuild.Shim.PropertiesFileGenerator.ToProjectData(IGrouping`2 projectsGroupedByGuid)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at SonarScanner.MSBuild.Shim.PropertiesFileGenerator.TryWriteProperties(PropertiesWriter writer, IEnumerable`1& allProjects)
   at SonarScanner.MSBuild.Shim.PropertiesFileGenerator.GenerateFile()
   at SonarScanner.MSBuild.PostProcessor.PostProcessor.GenerateAndValidatePropertiesFile(AnalysisConfig config)
   at SonarScanner.MSBuild.PostProcessor.PostProcessor.Execute(String[] args, AnalysisConfig config, IBuildSettings settings)
   at SonarScanner.MSBuild.BootstrapperClass.PostProcess()
   at SonarScanner.MSBuild.BootstrapperClass.<Execute>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SonarScanner.MSBuild.Program.<Execute>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SonarScanner.MSBuild.Program.<Execute>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SonarScanner.MSBuild.Program.<Main>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at SonarScanner.MSBuild.Program.<Main>(String[] args)
##[error]The process 'F:\agent\_work\_tasks\SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157\5.15.0\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe' failed with exit code 3762504530
Finishing: SonarQubeAnalyze

and this is all information I have. Maybe there is some switch I can turn on to find at least the path to see whats wrong.

Thanks for your help.

Colin · September 8, 2023, 5:25am

What’s the final one, and would you be able to extract it from the build agent for review?

msedi · September 8, 2023, 6:19am

The last one is 99 although there are in total 331. So they obviously not processed in numerical order. So if this is OK, the last one that is displayed is 99.json


08:07:37.047  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\98\Issues.json
08:07:37.062  The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: F:\agent\_work\2\.sonarqube\out\99\Issues.json

##[error]Unhandled Exception:

If it helps, I have uploaded the whole 99 folder. If you require more information - I’m available
99.zip (2.5 KB)

Additionally I have uploaded the last 331 from the numerical order, because it seems this is also the last created folder (by looking at the folder creation date).
331.zip (99.4 KB)

Colin · September 8, 2023, 7:39am

Thanks. I’m not quite sure where to go from here, so I’ll tag this for some experts.

gregory.paidis · September 14, 2023, 1:59pm

Hey @msedi ,

Share the Scanner for .NET verbose logs

Add /d:"sonar.verbose=true" to the…
- SonarScanner.MSBuild.exe or dotnet sonarscanner begin command to get more detailed logs
  - For example: SonarScanner.MSBuild.exe begin /k:"MyProject" /d:"sonar.verbose=true"
- “SonarQubePrepare” or “SonarCloudPrepare” task’s extraProperties argument if you are using Azure DevOps
  - For example:
```
- task: SonarCloudPrepare@1
    inputs:
      SonarCloud: 'sonarcloud'
      organization: 'foo'
      scannerMode: 'MSBuild'
      projectKey: 'foo_sonar-scanning-someconsoleapp'
      projectName: 'sonar-scanning-someconsoleapp'
      extraProperties: |
        sonar.verbose=true
```
The important logs are in the END step (i.e. SonarQubeAnalyze / SonarCloudAnalyze / “Run Code Analysis”)

Share the msbuild detailed logs

MsBuild.exe /t:Rebuild /v:d

or

dotnet build -v:d

msedi · September 14, 2023, 2:57pm

Thanks for your help.

Attached is the log of our whole build pipeline.

logs_8790.zip (3.6 MB)

I don’t know if this helps. If you need more, please just ask.

Thanks

gregory.paidis · September 18, 2023, 12:07pm

Hey again @msedi ,

Could you run the build part of your dotnet/msbuild step with verbosity set to Detailed and re-send us the logs?

For MSBuild you need: /v:d
For dotnet you need -v:d

For reference, I already opened a ticket on the appropriate a repo, where you can track it.
I am just curious to see which path is the culprit, which might make our verification more accurate when implementing said ticket

Thanks again,
Greg

msedi · September 19, 2023, 6:14am

Hi Gregory,

sorry, that was my fault. While playing around I accidently took the wrong pipeline output.
Attached is the log output with -v:d.

Best
Martin

9_Build solution IPIPE.sln.txt.zip (23.0 MB)

msedi · September 22, 2023, 1:51pm

Hi Gregory,

just for you rinformation. It seems the problem was in the excludes.
I was using

sonar.exclusions=...Tables/,Tools/,tools.p/,pipelines/,documentation/

because I wanted to exclude all files unterneath the given folders. When removing the items it worked again.

To make it work I thought the following line works, but it doesn’t

sonar.exclusions=Tables/**,Tools/**,tools.p/**,pipelines/**,documentation/**

What would be the correct way to exclude everything unterneath a folder?

msedi · September 28, 2023, 1:11pm

Hi Gregory,

unfortunately I thought the problem is the wrong path filters. But this was not the case. So I’m struggling now since 22 days with the problem without any solution. Even our main IT department cannot solve it, so it would be nice if we could work on this.

gregory.paidis · October 2, 2023, 2:42pm

Hey @msedi ,

I apologize for the delay.
Unfortunately, after looking into it a bit more, there doesn’t seem to be an easy solution I can implement right now.

I will inform my team and try to prioritize it as much as possible during upcoming work on the scanner side.

Thanks for raising this and sorry again for my slow reply,
Greg

msedi · October 2, 2023, 4:17pm

Gregory, no problem. Is there anything I can do from my side? I think the only problem is that I cannot find out which path causes the problem.

The other side is, even if you can solve something on your side, our company installed SonarQube might not be updated accordingly, so solving the problem on my side would help me most at the moment.

Thanks for your help
Martin

gregory.paidis · October 3, 2023, 12:13pm

Hey again Martin,

Based on your error stack trace, what seems to be the problem is a call to new FileInfo.

As you can see in the constructor exceptions, there is some detection of illegal characters that throws this stack trace.

You could try to find all the files that are processed during the build process from the verbose logs and write a script to check them.

Note: This seems to be happening only in .NET Framework and early .NET Core versions.

Extra note: If you really want to go in depth, you can also check the source code of the method that is throwing inside the runtime.