Exclusion for all XML files doesn't appear to be working for total lines of code calculation

I’m using SonarCloud with Azure DevOps on a predominantly C# codebase with supporting web files. I’m trying to exclude all XML files via the UI (Administration > General Settings > Analysis Scope > Source File Exclusions) using **/*.xml but in the Overview I’m seeing this:

I can see that there was an issue with exclusions set in the UI not being taken into account back in 2020 but that got fixed. Grateful for any pointers as to what I might be doing wrong. Thanks!

Hi,

Welcome to the community!

There’s a hierarchy to settings, with those set via the UI at the bottom.

Can you check your Administration → Background Tasks → [row dots menu] → Show SonarScanner Context to see what exclusions were actually in force for the analysis?

 
Ann

Many thanks for your reply Ann! Exclusions in force as follows:

sonar.exclusions=**/*.sql,**/*.css,**/vendor/**/*.*,**/jquery*.*,**/Databases/Vivantio.*/**/*.*,**/mxgraph/**/*.*,**/DeploymentScripts/**/*.*,**/FileStorage/**/*.*,**/LegacyReports/**/*.*,**/WcfFileStore/**/*.*,**/MvcUI/**/*.js,**/MvcSelfService.Advanced/**/*.js,**/MvcMobile/**/*.js,**/MvcPlatformManagement/**/*.js,**/*publish.xml,**/*.min.js,**/*.xml

As far as I can tell these are all being set in the UI.

Hi,

Thanks for checking that. Could you provide your full analysis log, please?

The analysis / scanner log is what’s output from the analysis command. Hopefully, the log you provide - redacted as necessary - will include that command as well.

This guide will help you find them.

 
Thx,
Ann

Many thanks Ann! I have the debug log but it’s 63k lines long and thus difficult to redact and I’d rather not post it publicly. Would it be possible to send it to you privately please? I am of course happy for salient extracts to be posted here to help anyone in the future.

1 Like

Hi,

Which scanner are you using?

 
Ann

Hi Ann, it’s SonarScanner for MSBuild 5.15 in an Azure DevOps pipeline using the SonarCloudPrepare@1 task.

Hi,

And your other exclusions are working as expected? (Sorry to drag this out one question at a time. I keep not having the a-ha moment I’m expecting.)

 
Ann

No problem! At a very broad level there are 20k of CSS files being reported in the overview which I’m not expecting to see based on the exclusion filter. The more granular exclusions do seem to be working.

Hi,

This is fascinating. Does “working granular exclusions” include **/*publish.xml? Also, what about the **/*.sql exclusion? At this point I’m guessing it’s failing?

And finally, I’m well beyond my expertise at this point, so I’m going to flag this for more expert eyes.

 
Ann

Hi folks,

Could it be that the not-excluded XML files are classified as part of a test project? In this case, the way to exclude them would be to set the test-specific property: sonar.test.exclusions.

Thanks for that thought @Julien_HENRY. We do have test projects but the files are predominantly C#.

I’ve been looking again at the SonarCloudAnalyze output generated by Azure DevOps (with sonar.verbose=true). I can see that the C# files I have looked at get 7 ‘mentions’ whilst the XML files I have checked only get 2 ‘mentions’. I can see that the C# files are getting indexed but the XML files are not. Is that the indication that the exclusions are working and the issue is with the final Lines of Code calculation?

Apologies @ganncamp - just realised that I hadn’t answered your questions. As far as I can tell, in the SonarCloudAnalyze output generated by Azure DevOps everything we want to exclude in terms of full filenames appears only twice (including **/*publish.xml and **/*.sql) whilst what we do want to include appears seven times. So that bit seems to be working.

The problem seems to be in the generation of the final Lines of Code calculation which appears to be calculating 30k too high due to CSS and XML. This brings us to a total of 494k lines of code which is perilously close to our 500k billing allowance hence my desire to get loc down. Cheers!

1 Like

Just to double-check, can you try excluding CSS and XML for both production and test code:

Thanks for that suggestion @Julien_HENRY - I have added those exclusions and run the analysis again but still no change:

image

Quick update: mysteriously XML has dropped from 10 k to 4.4 k:


Is there any way to trace what changed?

1 Like

I don’t think there is a way today to drilldown from the UI to find files by language.

I just realized something: what SonarCloud will consider as XML file is controlled by the file suffixes property, and by default it is .xml,.xsd,.xsl,.config.

So if you really want to exclude all “XML” files, you should configure exclusions like:

sonar.exclusions=**/*.xml,**/*.xsd,**/*.config,**/*.xsl
sonar.test.exclusions=**/*.xml,**/*.xsd,**/*.config,**/*.xsl
1 Like

Great thought @Julien_HENRY - I’ve just tested and those extra exclusions have WORKED! :grin:

In a similar vein I wondered if SonarCloud has a more liberal interpretation of CSS and it DOES! :grin:

Excluding **/*.scss and **/*.sass has almost eliminated the CSS element. Weirdly, there are still 919 lines of CSS which I can live with of course, but would be interested to know where they are coming from. If CSS is embedded in HTML files will it pick those up?

Many thanks for all the responses to this - hopefully will be of use for others.

Have you also excluded .less files?

I don’t think there is a way to search files by language on the server side.

But on the scanner side, if you enable verbose logs, you should see the list of indexed files with their assigned language.

If CSS is embedded in HTML files will it pick those up?

I don’t think so.

.less was the culprit! Adding that as an exclusion has completely removed CSS from the analysis :smile:

2 Likes