Tips for speeding up analysis time?

SonarQube Cloud
, ,
24

You can check if there are any " -verbosity: level" or "-v: level" flag passed in the msbuild command for your main build

25

@mickaelcaro No, Please see attached code

    - task: VSBuild@1
        enabled: true
        displayName: 'Sitecore.Website build'
        inputs:
          solution: '**/src/Project/xyz/code/xyz.Sitecore.Website.csproj'
       msbuildArgs:
 '/p:DeployOnBuild=true /p:DeployDefaultTarget=WebPublish /p:WebPublishMethod=FileSystem  /p:publishUrl="$(build.sourcesdirectory)\071DPLY\\" /p:BuildProjectReferences=true /p:DebugSymbols=false /p:DebugType=None /p:outputpath="$(build.sourcesdirectory)\071DOUTFLDR\\"'
          platform: '$(buildPlatform)'
          configuration: '$(buildConfiguration)'
26

My Azure VMs are in West Europe

See logs attached. sonarcloud_prepareanalysis_msbuild.zip (1.4 MB)

Background task example: AXUiDNT1oV2Q5xcOwqiF, yet this completed in 37s. Just to re-iterate in case I wasn’t clear before, the Prepare Analysis and Run Code Analysis are not my concern, instead my concern is that the MSBuild task is 10x slower when the SonarCloud is enabled.

27

And to be noted here just to make it clear : We are installing targets file during the Prepare step to hook up into the build step, so there’s definitely an overhead to expect, though 10x time slower is not.

29

@Olivier_Schmitt @Christophe_Havard @mickaelcaro - I think this would probably helpful in drill down above issue I was reading one of @Olivier_Schmitt comment so its worth sharing

We have 190 Projects right now in the solution and our build time increased as number of projects getting increased I’m suspecting(See below attached image)
was 250k +/- 6 months ago, now is 2250k +/- now

image (1)
image (1)2392×822 64 KB

  • the SonarCloud scanner scans every project = 190 analysis to perform
  • each analysis takes 1 minute to scan the code which is quite fast (but can be improved if the scope is reduced to the minimum) = 190 minutes of code analysis. Even with a 30 seconds step, the build time will be multiplied by 8

To validate above theory I just change my build step to contain only one more project which has no reference to other project and it just took 32s Capture

Having said that what would you guys recommend and what should be minimum accepted time for scanning a project

Thanks in advance

31

Hi @Schumi

Could you please add in your msbuild task the following command :

/p:reportanalyzer=true > build.log

And post the report (i can PM you if you need) this will allow us to know which rule take the most time to be processed against your code.

Thanks in advance !

32

@mickaelcaro Thanks, Let me try this. Yes please PM me so that I can share the report with you

36

Hi @alexvaccaro. It’s not clear to me from this thread if you are in the same team with @Schumi or have a different topic altogether.

RE: msbuild, like @mickaelcaro said, most of our analysis runs during msbuild.

The analysis is done in two steps:

  • during the build - our native Roslyn analyzers are running for most of the rules (~250 in the default QP); also, UCFG files are created for the vulnerability injection analysis which is done in the SonarCloudAnalyze step
  • during the SonarCloudAnalyze step - our security engine runs over the UCFG files to find vulnerabilities - we currently have 12 rules that detect injection vulnerabilities

The msbuild debug logs don’t help that much, we’d need the msbuild /v:d /p:reportanalyzer=true > build.log logs. At the end of each project compilation, it will print out a list with each of our rules and how much time it took. Like this we can detect outliers, and also maybe projects that are outliers. We can then advise based on this info.

If you want, we can continue this topic in a private thread , like we did with @Schumi. Or we can continue here.

38

Hi @Andrei_Epure , thank you for getting back to me, I am not with Schumi.

I have been busy playing around with Azure VMs options for my build agents and I have found some gains in performance and I will be testing this in the coming days, but I will try your suggestion as well.

I would prefer a private thread so I can share my logs more freely.

39

Hi @alexvaccaro, I sent you a PM to continue there

40

Was there an answer to this one?

I have a similar issue; without SonarCloud a built on Azure DevOps hosted instances takes around 45 minutes including tests.

WITH SonarCloud: 130 minutes.

Now I am trying with self hosted agents:

WITHOUT SonarCloud: 27 minutes
WITH SonarCloud: 90 minutes

Any hints are appreciated.

1 Like
41

I am also interested in any hints for this problem

42

hello @gimlichael , @Vajda_Endre - as per our community guideline, please

Keep to one subject per topic
Most of the time, your question will deserve a new topic, containing information specific to your situation. If many topics should be consolidated into one, we’ll take care of it! In addition


  • Do not bump old posts and try to tag-on with a possibly related issue.
  • Do not divert a topic by changing it midstream

We’ve deployed fixes for the problems we found in what @Schumi and @alexvaccaro reported.

The analysis can behave differently for each project in terms of performance. This is why we need each problem to be on its own topic. We have a backlog of performance improvements to do (and we are doing them regularly). We are welcoming all feedback in order to be able to prioritize what’s most impactful, and to find unknown issues.

I am closing this topic as we’ve addressed the issues reported in it.

Please open new topics, and we can address your specific problems and provide tailored suggestions .

45

Hello

For users of SonarCloud and SonarQube 9.1+ you now have the ability to run the analysis in concurrent execution mode. This can significantly reduce your analysis time. Just set the following environment variable in the begin step:

SONAR_DOTNET_ENABLE_CONCURRENT_EXECUTION: true

We would love to hear if you have tried this, if you have please could you post a comment on this post and tell us how you got on?

Thanks

Tom

1 Like