which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension): 8.4
what are you trying to achieve : push sonar-scanner reports in parallel
what have you tried so far to achieve this: NA
I have around 400+ repository. If i scan them one by one it will take more time. I am thinking to scan repos in parallel using multiprocessing (batch of 10. client machine is 16 vCPUx64 GB RAM).
My question is, will sonarqube server support push of 10/20 analysis reports simultaneously?
The ability to manage Compute Engine performance is available as part of Enterprise Edition and above.
In Community Edition or Developer Edition, you are only able to run the Compute Engine with 1 worker; so you cannot process analysis reports in parallel on SonarQube server.
In Enterprise Edition, what you explain is actually possible as you can run up to 10 workers that can process analysis reports in parallel. To give you a complete overview, beyond this vertical scalability, the Data Center Edition allows spinning up to 10 application nodes in different hosts of a cluster, each with up to 10 workers.
As per my understanding , sonar-scanner connect to SQ server through web process and then report is passed to compute engine process for processing. correct me if i am wrong.
if multiple reports are sent by sonar-scanner in parallel then web process will keep them in queue and pass one report at a time to compute engine. Is it correct ?
These parameters are configuring the SonarQube’s Web HTTP server parameters, which a separate logical component from the Compute Engine. It has nothing to do with analysis report processing capacity. As described in the sonar.properties file:
# The maximum number of connections that the server will accept and process at any given time.
# When this number has been reached, the server will not accept any more connections until
# the number of connections falls below this value. The operating system may still accept connections based on the sonar.web.connections.acceptCount property. The default value is 50.
#sonar.web.http.maxThreads=50
# The minimum number of threads always kept running. The default value is 5.
#sonar.web.http.minThreads=5
# The maximum queue length for incoming connection requests when all possible request processing
# threads are in use. Any requests received when the queue is full will be refused.
# The default value is 25.
#sonar.web.http.acceptCount=25
Your understanding on how the communication flows between components is right, but again, your Compute Engine will always be restricted to processing 1 analysis report at a time with 1 single worker.
Thanks Daniel.
Based on above explaination, can i say that max 50 sonar-scanner instances can make connection to web process and push the analysis report and further these 50 analysis reports will be processed one at a time ?
I am unsure of what you are trying to achieve here. The number of web server connections include any HTTP(S) connection to the SonarQube Server: users browsing the UI on the browser, Web API calls, and of course analysers.
If you have 50, 75, 100 or 200 scanners sending their report at the same time, the Compute Engine will still “bottleneck” by processing one at a time.
To be technically precise and answer your question: no, the maximum scanner instances that will be able to connect at the same time will be 75, as you need to take into account the queue (acceptCount=25). But this is if no other connection is open with the server at that time.
Now again, this won’t play any real nor worrying role in your use case as you only have 1 CE worker, and not all scanner analysis will send their report at the exact same time. When it comes to the Web Server, the default settings should be fine.
To achieve your original goal of increasing parallel processing capacity of analysis reports by the server, you’d need to upgrade to at least Enterprise Edition and increase the number of workers.