Second sonar quality gate check is over 1000% slower than first for no apparent reason

Since I assumed those *.Designer.cs files get automatically excluded, I just checked it. According to sonar-.net-documentation/appendix-2.md at master · SonarSource/sonar-.net-documentation · GitHub documentation xxx.Designer.cs files are automatically excluded.

I already have spend quite some time in preparing some log files with log file references to a fast and two slow runs on debug and trace level some time ago. Based on some earlier requests via communication via email which lead me then finally to this forum. I explictly had to repalce some sensitive strings that were included, so it was just not copy and paste.

I don’t want to post them here. If those logs are fine for you I could send them to you via email.

Hi,

I’ve initiated a private thread to transmit those logs in.

And I think it’s worth noting those docs you cited are in an archived repo - i.e. not incurrent use - and haven’t been updated since 2016. Since that exclusion isn’t in the current docs, I wouldn’t count on it being there.

 
Ann

I found a workaround that speeds it up dramatically: From 14-15min to 4s for a PR and 30s for branches => So it’s about 20000% faster in some cases.

The workaround is to specifiy project specific source and test folders in the sonar scanner begin call

/d:<guid of prod project>.sonar.sources=SourceFolder1/,SourceFolder2/
/d:<guid of test project>.sonar.tests=SourceFolder1/,SourceFolder2/

where the overall structure is at our project something like this:

MainFolder/ProdProject/SourceFolder1/sub/folders/SomeFile1.cs
MainFolder/ProdProject/SourceFolder2/sub/folders/SomeFile2.cs
MainFolder/TestProject/SourceFolder1/sub/folders/SomeFile1.cs
MainFolder/TestProject/SourceFolder2/sub/folders/SomeFile2.cs

The duplication detection is now also reporting much less lines with this workaround and not anymore the 400k as before.

Very helpful is in this regards is the generated .sonarcube/out/sonar-project.properties file which was containing for each project a <project.gui>.sonar.sources or <guid of prod project>.sonar.tests property with a HUGE list of long absolute file names. Those file paths explicitly listed as source included HUGE coverage.opencover.xml files and some other large resource files that are not relevant for sonar.

Note that the workaround is not perfect, as when new source folders get added they will not be analysed by sonar but will be silently ignored.

So in summary I think there are multiple bugs:

  • SonarScanner for MSBuild 5.10 includes too many not relevant files and possibly creates a too large source list for sonar
  • SonarCube 9.8.0.63668 Developer does not apply it’s own excludes to ignore not relevant data to speed up file movement detection
  • SonarCube 9.8.0.63668 Developer may be ineffiecent in file movement detection when dealing with large source lists
  • SonarCube 9.8.0.63668 Developer can be as initially described much slower on second analysis for no good reason (As a user I want to see how the PR compares to the reference branch and not to the previous analysis. There were no freshly closed/opened issues so there should have been no reason for the sonar quality gate check to compare to the reference branch for both first and second quality gate check).

Hi @florian4s,

I read all the thread. I think it’s important to first give you the knowledge about how things work::

  • The SonarScanner for MSBuild includes files based on what is in .csproj/.sln files. So if there are in your VisualStudio project, they will be in your analysis, and the scanner can’t decide whether or not a project file is relevant, ie. they all are from a scanner perspective.
  • The step Detect file moves doesn’t execute during the first analysis (ie. there is no baseline)
  • The step Detect file moves doesn’t execute on PR analysis (there is another dedicated one instead). So you’re obviously analyzing your code as a branch, right? If you are analyzing a PR, you should make sure your CI (Jenkins if I get it right) runs the analysis as a PR, not a branch.
  • Regardless of the comparison method (reference branch, previous analysis, etc.), at the end of the day, SonarQube has to compare a previous report to the current one. It necessarily involves executing Detect file moves, we can’t infer file move/rename in any other way. So it’s not a bug that it executes, it’s by design.
  • The step Detect file moves has to compute the difference between 2 sets of files. This takes time when files content are overly long, like an XML coverage report for example. This is not a bug, performances on such huge files can’t be easily improved, those files should not be in analysis scope in the first place.

In the end, you had the good approach: to scope the analysis to only files that are relevant. However, instead of hardcoding sonar.sources and sonar.tests, you’d be better using sonar.exclusions to exclude irrelevant files, as @ganncam mentioned. Indeed as you spotted “when new source folders get added they will not be analyzed by sonar but will be silently ignored”, this will not happen if you use exclusions.
You could also choose to restructure your project to not have those files there in the first place.

So as a conclusion, there is no bug or misbehavior here, the solution is to not have huge files in the analysis scope, especially if this is not source code (XML/JSON/SQL/… exports, generated code, etc.)

I hope it makes sense.

Cheers,
Antoine

1 Like

Hello @Antoine

Thanks for taking the time to read the whole post and for taking the time to composing a reply:

  1. About SonarScanner for MSBuild: The coverage and other test result files are not in the project source. We also don’t have special code that creates them. We have normal csharp xunit test projects for the test code. I think the coverage file is even generated by/for the sonar scanner. So I would say it is a bug they are included. There are different test frameworks in csharp, could it be that sonar scanner can’t handle xunit projects yet?

  2. About detect file moves point 1: Agree there should be no Detect file moves if it is the first build on a branch without a baseline configured to compare against

  3. For point 2 about Detect file moves. For the explicit tests I made, I used a test branches. We had the performance issue however also for PRs.

  4. If I understand your third point about the step Detect file moves corectly it has to run when I start a branch build with a reference branch configured. => So it also has to start for the first build of that branch as there is another branch to compare it to. => It is strange that it is not slow for the first build of a branch when there is a reference branch configured!

  5. Sadly the excludes do not help with the performance. Wild guess: Maybe the excludes are not applied for the file list used by the file movement detection. Or another wild guess: The long list of included source file paths is the source of the slow down (e.g. because it compares each file against the huge list).

Florian, it all boils down to this fact: why are those files included in the first place? I doubt that this is a bug but there must be an explanation. Let’s find out.

Could you please turn on debug mode for analysis (add the /d:sonar.verbose=true parameter to your begin step) and share those logs with me? That way, I could try to understand what happening. Note that I need the full analysis logs: begin, build and end.

Cheers
Antoine

Hello @Antoine ,

how can I send the log files to you?

I don’t want to post them in this public thread.

Hello @florian4s,

This is a Community forum and your case could be useful to others as well. Of course, I understand that you don’t want to leak any sensitive information. To comply with these 2 concerns, could you please redact what’s necessary from your logs and still share them here? That would be the ideal solution.

Cheers,
Antoine

1 Like

Hello @Antoine ,
here are the log lines that I think are relevant with edits to remove sensitive data.

Lines from begin part in which sonar.cs.opencover.reportsPaths is used to pass in the location of the coverage files:

C:\Jenkins_Home\workspace\demo-branch>dotnet "C:\Jenkins_Home\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\Sonar_.NET_6.0\SonarScanner.MSBuild.dll" begin /k:SolutionName /n:SolutionName /d:sonar.verbose=true /v: /d:sonar.cs.opencover.reportsPaths=SolutionName/**/TestResults/**/coverage.opencover.xml /d:sonar.coverage.exclusions=**/SolutionName/*Tests/**/Test*.cs,**.xaml.cs,**/SomeMoreExplictExcludes/Example.cs 
SonarScanner for MSBuild 5.10
Using the .NET Core version of the Scanner for MSBuild
Default properties file was found at C:\Jenkins_Home\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\Sonar_.NET_6.0\SonarQube.Analysis.xml
Loading analysis properties from C:\Jenkins_Home\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\Sonar_.NET_6.0\SonarQube.Analysis.xml
sonar.verbose=true was specified - setting the log verbosity to 'Debug'
16:08:32.075  Updating build integration targets...
...
16:08:33.192  Pre-processing succeeded.

Actual build:

C:\Jenkins_Home\workspace\demo-branch>"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin\MSBuild" SolutionName/SolutionName.sln /p:Configuration=Debug 
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
Build started 3/13/2023 4:08:33 PM.
Project "C:\Jenkins_Home\workspace\demo-branch\SolutionName\SolutionName.sln" on node 1 (default targets).
...
Build succeeded.
...
Time Elapsed 00:02:37.18

The end phase with several references to the coverage file and other test result files

C:\Jenkins_Home\workspace\demo-branch>dotnet "C:\Jenkins_Home\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\Sonar_.NET_6.0\SonarScanner.MSBuild.dll" end 
SonarScanner for MSBuild 5.10
Using the .NET Core version of the Scanner for MSBuild
Post-processing started.
16:11:12.523  16:11:12.304  Uninstalling target: C:\Jenkins_Home\workspace\demo-branch\.sonarqube\bin\targets\SonarQube.Integration.targets
16:11:12.523  16:11:12.523  sonar.verbose=true was specified - setting the log verbosity to 'Debug'
16:11:12.524  Loading the SonarQube analysis config from C:\Jenkins_Home\workspace\demo-branch\.sonarqube\conf\SonarQubeAnalysisConfig.xml
...
16:11:12.787  Dumping content of sonar-project.properties
------------------------------------------------------------------------
...
test-project-guid.sonar.tests=\
...
"C:\\Jenkins_Home\\workspace\\demo-branch\\SolutionName\\SomeProjectTests\\TestResults\\some-guid\\coverage.opencover.xml",\
"C:\\Jenkins_Home\\workspace\\demo-branch\\SolutionName\\SomeProjectTests\\TestResults\\PROJECT-BUILD$_PROJECT-BUILD_2023-03-13_16_07_39\\In\\PROJECT-BUILD\\coverage.opencover.xml",\
"C:\\Jenkins_Home\\workspace\\demo-branch\\SolutionName\\SomeProjectTests\\TestResults\\SomeProjectTest_Results.xml"
...
sonar.cs.opencover.reportsPaths=SolutionName/**/TestResults/**/coverage.opencover.xml
sonar.coverage.exclusions=**/SolutionName/*Tests/**/Test*.cs,**.xaml.cs,...
...
project-guid.sonar.cs.analyzer.projectOutPaths=\
"C:\\Jenkins_Home\\workspace\\demo-branch\\.sonarqube\\out\\3"
project-guid.sonar.cs.roslyn.reportFilePaths=\
"C:\\Jenkins_Home\\workspace\\demo-branch\\.sonarqube\\out\\3\\Issues.json"
...
sonar.visualstudio.enable=false
...
16:11:28.098 DEBUG:   Test paths: some/file/paths/ExampleTest.cs, .. TestResults/some-guid/coverage.opencover.xml, TestResults/PROJECT-BUILD$_PROJECT-BUILD_2023-03-13_16_07_39/In/PROJECT-BUILD/coverage.opencover.xml, TestResults/SomeProjectTest_Results.xml
16:11:28.098 INFO:   Excluded sources: **/SomeFolder/*.*
16:11:28.098 INFO:   Excluded sources for coverage: **/SolutionName/*Tests/**/Test*.cs, **.xaml.cs, **/SomeMoreExplictExcludes/Example.cs 
16:11:28.098 INFO:   Excluded sources for duplication: **/*.*
...
16:11:29.400 DEBUG: 'SomeProjectTests\TestResults\some-guid\coverage.opencover.xml' indexed as test with language 'xml'
16:11:29.400 DEBUG: File SomeProjectTests/TestResults/some-guid/coverage.opencover.xml excluded for duplication
...
16:11:29.409 DEBUG: 'SomeProjectTests\TestResults\PROJECT-BUILD$_PROJECT-BUILD_2023-03-13_16_07_39\In\PROJECT-BUILD\coverage.opencover.xml' indexed as test with language 'xml'
16:11:29.409 DEBUG: File SomeProjectTests/TestResults/PROJECT-BUILD$_PROJECT-BUILD_2023-03-13_16_07_39/In/PROJECT-BUILD/coverage.opencover.xml excluded for duplication
16:11:29.422 DEBUG: 'SomeProjectTests\TestResults\SomeProjectTest_Results.xml' indexed as test with language 'xml'
16:11:29.422 DEBUG: File SomeProjectTests/TestResults/SomeProjectTest_Results.xml excluded for duplication
...
16:11:41.038 INFO: Sensor C# Properties [csharp] (done) | time=0ms
16:11:41.038 INFO: Sensor XML Sensor [xml]
16:11:41.039 INFO: Sensor XML Sensor [xml] (done) | time=1ms
16:11:41.039 INFO: Sensor Text Sensor [text]
16:11:41.039 INFO: x source files to be analyzed
...
16:11:41.415 DEBUG: 'SomeProjectTests/TestResults/SomeProjectTest_Results.xml' generated metadata as test  with charset 'UTF-8'
...
16:11:42.561 DEBUG: 'SomeProjectTests/TestResults/some-guid/coverage.opencover.xml' generated metadata as test  with charset 'UTF-8'
...
16:11:43.999 INFO: x/x source files have been analyzed
...
16:12:46.561 INFO: EXECUTION SUCCESS
...
16:12:47.088  Post-processing succeeded.

Also interesting that with our base branch using the workaround the test branch had now also a faster quality gate check. Even though the test brach I used for this logs again was unchanged and was not using the workaround. Probably because with the large coverage files being gone from the base branch the compare between test and base branch was thus faster.