SonarCloud analysis of same project taking longer than in Sonarqube CE

[gazal-k (Gazal)]

• ALM used - GitHub
• CI system used - Jenkins X
• Languages of the repository - Java

We have a pretty big java project with > 255k LOC.
We have a sonarqube installation on our k8s cluster within the same VPC as our CI system.
The analysis of the project in sonarqube take about 2.5 mins.

We’ve recently bought into sonarcloud and we like the additional features when compared to sonarqube, but a downside we’ve observed is that it takes about 9 mins for the analysis now.

Is this just a network thing? The fact that sonarcloud is quite disconnected from the CI system as when compared to something within the same VPC?

Is there any way to speed things up?

[Olivier_Schmitt]

Hello Gazal,

Welcome to the community support!

The network can have a dramatic impact on the overall analysis duration (scanner + SonarCloud processing to build the final the analysis report).

The bigger the analysis report is the longer it will take to upload to SonarCloud.

If you look at the scanner logs you should spot such lines:

INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=92ms
INFO: Load/download plugins (done) | time=338ms
…
INFO: Analysis report uploaded in 5046ms
INFO: ANALYSIS SUCCESSFUL, you can find the results at: xxx
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at xxx
INFO: Analysis total time: 21.015 s

SonarCloud is hosted in AWS Cloud in the Frankfurt region.

So, if your scanner tries to download/upload artefacts from a far distance it might take more than you were used to with your on premise architecture which is quite hard to beat on that matter (Internet is not as fast as a dedicated private local network unfortunately)

There are some optimizations depending on the analysis context (PR or long living branch), a plugin cache, … but it can not do everything.

So what do you see on the report upload side?

Best.

[gazal-k (Gazal)]

Thanks Olivier, we are in AWS Sydney region.

The plugins download does take much longer, about 40s I believe. And we can look at ways to cache it. But what is concerning is the analysis of source files itself.

On self managed SonarQube Community Edition Version 7.6 (build 21501) the logs look like this:

Java Main Files AST scan

4244 source files to be analyzed
311/4244 files analyzed, current file: ___.java
672/4244 files analyzed, current file: ___.java
986/4244 files analyzed, current file: ___.java
1235/4244 files analyzed, current file: ___.java
1667/4244 files analyzed, current file: ___.java
2035/4244 files analyzed, current file: ___.java
2517/4244 files analyzed, current file: ___.java
2902/4244 files analyzed, current file: ___.java
3280/4244 files analyzed, current file: ___.java
3481/4244 files analyzed, current file: ___.java
3967/4244 files analyzed, current file: ___.java
4244/4244 source files have been analyzed
Java Main Files AST scan (done) | time=118424ms

On sonarcloud it looks like this:

Java Main Files AST scan
4026 source files to be analyzed
Load project repositories
Load project repositories (done) | time=439ms
52/4026 files analyzed, current file: ___.java
164/4026 files analyzed, current file: ___.java
250/4026 files analyzed, current file: ___.java
374/4026 files analyzed, current file: ___.java
494/4026 files analyzed, current file: ___.java
620/4026 files analyzed, current file: ___.java
744/4026 files analyzed, current file: ___.java
861/4026 files analyzed, current file: ___.java
974/4026 files analyzed, current file: ___.java
1087/4026 files analyzed, current file: ___.java
1208/4026 files analyzed, current file: ___.java
1326/4026 files analyzed, current file: ___.java
1456/4026 files analyzed, current file: ___.java
1570/4026 files analyzed, current file: ___.java
1705/4026 files analyzed, current file: ___.java
1840/4026 files analyzed, current file: ___.java
1968/4026 files analyzed, current file: ___.java
2066/4026 files analyzed, current file: ___.java
2175/4026 files analyzed, current file: ___.java
2298/4026 files analyzed, current file: ___.java
2407/4026 files analyzed, current file: ___.java
2517/4026 files analyzed, current file: ___.java
2644/4026 files analyzed, current file: ___.java
2775/4026 files analyzed, current file: ___.java
2899/4026 files analyzed, current file: ___.java
3020/4026 files analyzed, current file: ___.java
3137/4026 files analyzed, current file: ___.java
3252/4026 files analyzed, current file: ___.java
3375/4026 files analyzed, current file: ___.java
3492/4026 files analyzed, current file: ___.java
3629/4026 files analyzed, current file: ___.java
3737/4026 files analyzed, current file: ___.java
3881/4026 files analyzed, current file: ___.java
4012/4026 files analyzed, current file: ___.java
4026/4026 source files have been analyzed
Java Main Files AST scan (done) | time=341156ms

[Olivier_Schmitt]

We had issue with the java analyzer:

Java analyzer v6.0: more accurate analysis, with fewer false-positives

Hello, I’m happy to announce that the v6.3 of the Java Analyzer is fixing most of the performance degradation introduced with the v6.0. According to the tests we run, analysis are almost back to the performance of the v5.14. If you don’t see any improvements while upgrading to v6.3, we would like to exchange with you. So don’t hesitate to report it on a dedicated thread. Here are some figures from our tests: [image] [image] [image] You can see from these graphs that the move to ECJ was no…

But it should have been fixed with the deployment of the v6.3! (which is deployed: https://sonarcloud.io/api/plugins/installed)

When did you perform this analysis?

BTW, the analysis is not really the same between SonarQ 7 and SonarCloud (Sonar security).

You might compare things which are not really comparable.

Finally, I noticed the number of Java files is not the same.

Is the code base really the same?

Did you compare the report upload times?

Best.

[gazal-k (Gazal)]

Thanks Olivier,

Yes the codebase isn’t exactly the same. We’re temporarily running Sonarcloud on an outdated fork of the repo.

The Sonarcloud analysis logs I shared was from yesterday, so I’m guessing it used 6.3.

I understand if the analysis is taking longer because the checks are more rigorous. Just wanted to double check that it’s not performance issues with the analyser.

[gazal-k (Gazal)]

The report upload times are different as expected. But the additional 7 secs is ok.

Sonarcloud:

Analysis report generated in 804ms, dir size=47 MB
Analysis report compressed in 6892ms, zip size=19 MB
Analysis report uploaded in 7581ms

I see what you mean by the analysis not being completely comparable. The size of the report is smaller for a more updated (and consequently larger) codebase

Analysis report generated in 1559ms, dir size=28 MB
Analysis report compressed in 5190ms, zip size=14 MB
Analysis report uploaded in 597ms

[gazal-k (Gazal)]

SonarCloud analysis of same project taking longer than in Sonarqube CE

Thanks Olivier, Yes the codebase isn’t exactly the same. We’re temporarily running Sonarcloud on an outdated fork of the repo. The Sonarcloud analysis logs I shared was from yesterday, so I’m guessing it used 6.3. I understand if the analysis is taking longer because the checks are more rigorous. Just wanted to double check that it’s not performance issues with the analyser.

Hi, can someone from SonarSource please confirm whether the difference in time taken for analysis between Sonarqube and SonarCloud is expected? SonarCloud seems to take more than twice as long.