Dotnet scanner exclusions not working

SonarQube Cloud
, ,
1

The DotNet scanner does not appear to be honouring exclusions.

Scanner version:

SonarScanner for MSBuild 10.1.2
....
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.11 Eclipse Adoptium (64-bit)
INFO: Windows 11 10.0 amd64

I have set up my XML configuration as follows:

<?xml version="1.0" encoding="utf-8" ?>
<SonarQubeAnalysisProperties xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sonarsource.com/msbuild/integration/2015/1">
    <Property Name="sonar.exclusions">
        XXXXX.Database/Data Generators/**/*;
        XXXXX.Database/Scripts/**/*;
        XXXXX.Database/Intergration/**/*;
    </Property>
</SonarQubeAnalysisProperties>

(I have also tried putting them all on one line.)

The log file reports the following exclusions:

INFO:   Excluded sources: **/build-wrapper-dump.json, XXXXX.Database/Data Generators/**/*;
XXXXX.Database/Scripts/**/*;
XXXXX.Database/Intergration/**/*;, coverage.net472.xml

However later in the log file I can see the following messages relating to files that should be excluded.

WARN: Unable to fully parse: C:/Users/AA/projects/XXXXX/[source-control-root-folder]/XXXXX.Database/Data Generators/Unseed Base Data.sql
WARN: Parse error starting from line 3
WARN: 
WARN: 
WARN: Unable to fully parse: C:/Users/AA/projects/XXXXX/[source-control-root-folder]/XXXXX.Database/Data Generators/Unseed Base Data.sql
WARN: Parse error starting from line 6

Note: [source-control-root-folder] isn’t really formatted like this, I’m showing where the project is cloned to in the folder structure.

If I check in the Sonar UI, all of the Analysis options are empty. The exclusions fields are not populated.

If it makes any difference I am working on a branch, could that be why the server settings are not getting set in the UI as well ?

Other than that, I’m not sure what else to try.

2

Update..

I am now trying my exclusions as follows:

<?xml version="1.0" encoding="utf-8" ?>
<SonarQubeAnalysisProperties xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.sonarsource.com/msbuild/integration/2015/1">
    <Property Name="sonar.exclusions">**/XXXXX.Database/Data Generators/*.sql;**/XXXXX.Database/Data Generators/**/*;**/XXXXX.Database/Scripts/*.sql;**/XXXXX.Database/Scripts/**/*;**/XXXXX.Database/Integration/*.sql;**/XXXXX.Database/Integration/**/*</Property>
</SonarQubeAnalysisProperties>

I can still see in my log file entries like this:

WARN: Unable to fully parse: C:/Users/AA/projects/XXXXX/[source-control-root-folder]/XXXXX.Database/Integration/Seed Data.sql

WARN: Parse error starting from line 6

The glob pattern **/XXXXX.Database/Integration/*.sql and **/XXXXX.Database/Integration/**/* should ignore these files.

I suspect as the files have a space in them, this is confusing things. I did test this on a glob pattern tester and it showed the files were included.

I really need to be not analysing these files.

I also have no idea why they are causing parse errors - there is nothing unusual about the files - they just contain SQL.

3

Hi,

For the SonarScanner for .NET, exclusions aren’t applied during CI-side analysis - so you’ll still see excluded files showing up in the logs as being processed. For the SS4.NET, exclusions are only applied server-side after all the heavy lifting is done.

With the exception of project name and key, analysis parameters aren’t saved into the DB. So exclusions you set in the pipeline will never show up in the UI.

Would it help to be able to exclude the project from analysis? (I believe that exclusion would be applied during CI-side analysis, but I wouldn’t swear to it in court.)

 
Ann

5

Hi,

Okay, I’ve got some new (to me) information.

It seems that the restriction on CI-side exclusions only applies to actual .NET files: .cs and .vb. Exclusions of other files are handled the normal way.

So it looks like your exclusion isn’t being picked up. Can you share a debug log?

Share the Scanner for .NET verbose logs

  • Add /d:"sonar.verbose=true" to the…
    • SonarScanner.MSBuild.exe or dotnet sonarscanner begin command to get more detailed logs
      • For example: SonarScanner.MSBuild.exe begin /k:"MyProject" /d:"sonar.verbose=true"
    • “SonarQubePrepare” or “SonarCloudPrepare” task’s extraProperties argument if you are using Azure DevOps
      • For example:
        - task: SonarCloudPrepare@3
    inputs:
      SonarCloud: 'sonarcloud'
      organization: 'foo'
      scannerMode: 'dotnet'
      projectKey: 'foo_sonar-scanning-someconsoleapp'
      projectName: 'sonar-scanning-someconsoleapp'
      extraProperties: |
        sonar.verbose=true
  • The important logs are in the END step (i.e. SonarQubeAnalyze / SonarCloudAnalyze / “Run Code Analysis”)

Share the msbuild detailed logs

MsBuild.exe /t:Rebuild /v:d

or

dotnet build -v:d

 
Ann

6

Hi Ann,

The log file generated is over 300,000 lines so it will be quite hard to share it.

To give a more concrete example, in the project there is a 3rd party javascript library called “ace”

When I start to analyse that project I can see the following:

12:03:32.547 INFO: Indexing files of module 'CompanyName.Application.Web'
12:03:32.547 INFO:   Base dir: XXXXX\Application.Web

Then I get a DEBUG entry that contains a comma separated list of every single file in that folder which includes entries for scripts\ace. This is a huge list so I cannot reproduce it all here unfortunately e.g.

Scripts/ace/ace.js, Scripts/ace/ext-beautify.js, Scripts/ace/ext-chromevox.js, Scripts/ace/ext-elastic_tabstops_lite.js, Scripts/ace/ext-emmet.js, Scripts/ace/ext-error_marker.js, Scripts/ace/ext-keybinding_menu.js, Scripts/ace/ext-language_tools.js, Scripts/ace/ext-linking.js, Scripts/ace/ext-modelist.js, Scripts/ace/ext-old_ie.js, Scripts/ace/ext-searchbox.js, Scripts/ace/ext-settings_menu.js, Scripts/ace/ext-spellcheck.js, Scripts/ace/ext-split.js, Scripts/ace/ext-static_highlight.js, Scripts/ace/ext-statusbar.js, Scripts/ace/ext-textarea.js, Scripts/ace/ext-themelist.js, Scripts/ace/ext-whitespace.js, Scripts/ace/keybinding-emacs.js, Scripts/ace/keybinding-vim.js, Scripts/ace/mode-abap.js, Scripts/ace/mode-abc.js, Scripts/ace/mode-actionscript.js, Scripts/ace/mode-ada.js, Scripts/ace/mode-apache_conf.js, Scripts/ace/mode-applescript.js, Scripts/ace/mode-asciidoc.js, Scripts/ace/mode-assembly_x86.js, Scripts/ace/mode-autohotkey.js, Scripts/ace/mode-batchfile.js, Scripts/ace/mode-c9search.js, Scripts/ace/mode-cirru.js, Scripts/ace/mode-clojure.js, Scripts/ace/mode-cobol.js, Scripts/ace/mode-coffee.js, Scripts/ace/mode-coldfusion.js, Scripts/ace/mode-csharp.js, Scripts/ace/mode-css.js, Scripts/ace/mode-curly.js, Scripts/ace/mode-c_cpp.js, Scripts/ace/mode-d.js, Scripts/ace/mode-dart.js, Scripts/ace/mode-diff.js, Scripts/ace/mode-django.js, Scripts/ace/mode-dockerfile.js, Scripts/ace/mode-dot.js, Scripts/ace/mode-drools.js, Scripts/ace/mode-eiffel.js, Scripts/ace/mode-ejs.js, Scripts/ace/mode-elixir.js, Scripts/ace/mode-elm.js, Scripts/ace/mode-erlang.js, Scripts/ace/mode-forth.js, Scripts/ace/mode-fortran.js, Scripts/ace/mode-ftl.js, Scripts/ace/mode-gcode.js, Scripts/ace/mode-gherkin.js, Scripts/ace/mode-gitignore.js, Scripts/ace/mode-glsl.js, Scripts/ace/mode-gobstones.js,

These must be an append to the base path e.g.: XXXXX\Application.Web\Scripts/ace/ace.js

Note the different slash directions between the base dir log entry and the list of files in the DEBUG entry.

Next I can see my exclusions:

12:03:32.549 INFO:   Excluded sources: **/build-wrapper-dump.json, **/Application.Database/Data Generators/*.sql;**/Application.Database/Data Generators/**/*;**/Application.Database/Scripts/*.sql;**/Application.Database/Scripts/**/*;**/Application.Database/Integration/*.sql;**/Application.Database/Integration/**/*;**/Application.Web/Scripts/ace/*;**/Application.Web/Scripts/ace/**/*;**/Company.Application.Web.Admin/Content/**/*;**/Company.Application.Entities/**/*;**/Application.Web/Content/*;**/Application.Web/Content/**/*, coverage.net472.xml

You can see here that I exclude **/Application.Web/Scripts/ace/*;**/Application.Web/Scripts/ace/**/*.

Later in the log you can see that this folder starts getting processed:

12:03:33.690 DEBUG: 'Application.Web/Scripts/ace/ace.js' generated metadata with charset 'UTF-8'
12:03:33.691 DEBUG: Average line length for Application.Web/Scripts/ace/ace.js is 33
12:03:33.692 DEBUG: File Application.Web/Scripts/ace/ace.js was excluded because it looks like a bundle. (Disable detection with sonar.javascript.detectBundles=false)
12:03:33.692 DEBUG: 'Application.Web/Scripts/ace/ace.js' excluded by org.sonar.plugins.javascript.filter.JavaScriptExclusionsFileFilter
12:03:33.704 DEBUG: 'Application.Web/Scripts/ace/ext-beautify.js' generated metadata with charset 'UTF-8'
12:03:33.704 DEBUG: Average line length for Application.Web/Scripts/ace/ext-beautify.js is 23
12:03:33.704 DEBUG: 'Application.Web/Scripts/ace/ext-beautify.js' indexed with language 'js'
12:03:33.714 DEBUG: 'Application.Web/Scripts/ace/ext-chromevox.js' generated metadata with charset 'UTF-8'

And this:

12:04:05.405 DEBUG: Analyzing file: file:///XXXXX/Application.Web/Scripts/ace/theme-chaos.js
12:04:05.406 DEBUG: Analyzing file "XXXXX/Application.Web/Scripts/ace/theme-chaos.js"
12:04:05.406 DEBUG: Parsing XXXXX/Application.Web/Scripts/ace/theme-chaos.js with typescript-eslint/parser
12:04:05.421 INFO: Written udg to file:  XXXXX/.sonarqube/out/.sonar/mod3/architecture/js/_________Symphony_Web_Scripts_ace_theme-chaos_js.udg
12:04:05.425 DEBUG: Saving issue for rule S1516 on file Application.Web/Scripts/ace/theme-chaos.js at line 5
12:04:05.426 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/theme-chaos.js at line 153
12:04:05.428 DEBUG: Not enough content in 'Scripts/ace/theme-chaos.js' to have CPD blocks, it will not be part of the duplication detection
12:04:05.428 DEBUG: Cache entry created for key 'jssecurity:ucfgs:SEQ:10.23.0.32711:AAAAA_2_AAAAA:Application.Web/Scripts/ace/theme-chaos.js' containing 1 file(s)
12:04:05.429 DEBUG: Cache entry created for key 'jssecurity:ucfgs:JSON:10.23.0.32711:AAAAA_2_AAAAA:Application.Web/Scripts/ace/theme-chaos.js'
12:04:05.429 DEBUG: Cache entry created for key 'js:filemetadata:10.23.0.32711:AAAAA_2_AAAAA:Application.Web/Scripts/ace/theme-chaos.js'
12:04:05.430 DEBUG: Cache strategy set to 'WRITE_ONLY' for file 'Application.Web/Scripts/ace/mode-vbscript.js' as current analysis requires all files to be analyzed
12:04:05.430 DEBUG: Analyzing file: file:///XXXXX/Application.Web/Scripts/ace/mode-vbscript.js
12:04:05.431 DEBUG: Analyzing file "XXXXX/Application.Web/Scripts/ace/mode-vbscript.js"
12:04:05.431 DEBUG: Parsing XXXXX/Application.Web/Scripts/ace/mode-vbscript.js with typescript-eslint/parser
12:04:05.472 INFO: Written udg to file:  XXXXX/.sonarqube/out/.sonar/mod3/architecture/js/_________Symphony_Web_Scripts_ace_mode-vbscript_js.udg
12:04:05.477 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 4
12:04:05.477 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 5
12:04:05.478 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 7
12:04:05.478 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 9
12:04:05.478 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 181
12:04:05.478 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 182
12:04:05.478 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 183
12:04:05.478 DEBUG: Saving issue for rule S3504 on file Application.Web/Scripts/ace/mode-vbscript.js at line 185

So my exclusion isn’t being recognised, or is not applied at this point.

I’ve tried various glob patterns, different slash directions (etc) but nothing seems to remove this.

The problem is that my scan time is already pretty long and scanning these files is adding to that time.

Can you spot anything I have done wrong?

Maybe the exclusions only apply after the base directory so I only need /Scripts/ace ?
Or could they be case sensitive?

Thanks

7

Just to add, my current scan time for this project on a compute optimised machine is around 11 minutes.

To scan that file (which I’m not interested in) takes 30 seconds, nearly 5% of the total scan time. I also have other files which have the same problem. If I can exclude them all in the CI scan that will save a lot of time where developers are waiting for the scan results.

8

Hi,

I should have picked up on it the first time, but when I looked at your exclusions in the log format I realized that they’re semicolon-delimited. Analysis is expecting a comma-delimited list. I suspect that^ is being read as one long path with semicolons in it.

 
HTH,
Ann

9 
while (true)
{
    Console.WriteLine(":person_facepalming:");
}

:person_facepalming::person_facepalming::person_facepalming::person_facepalming:

It would appear to have been exactly that :upside_down_face:.

1 Like