We’re running SonarQube in Kubernetes and would like to use managed services of our Cloud provider as much as possible for stateful data, i. e. a managed PostgreSQL server and a managed Elasticsearch cluster.
Effectively, I’d like to only run stateless parts of SonarQube in Kubernetes and let the Cloud provider take care of “the hard stuff”, i. e. stateful databases.
Are you facing some specific problems with the way Elasticsearch is currently handled by SonarQube in Kubernetes?
Or does your request comes from a wish to rationalize the way you operate all your services?
It’s the latter: We want our pods in the Kubernetes cluster to be stateless and keep all databases out of the Kubernetes cluster (i. e. use the hosted services offered by our cloud platform).
With Elasticsearch “living” inside the same container as SonarQube itself, it could potentially be corrupted if the container is being stopped (for whatever reason). Even if Elasticsearch is not the system of record in this case, it would be one thing less to worry about when using the SonarQube Helm chart and running SonarQube in Kubernetes.
We would like to run SonarQube on our GCP Autopilot (a managed Kubernetes cluster) cluster. Which does not allow privileged Pods.
However, these are required to change (most) Linux Kernel settings. Elasticsearch requires a change of the vm.max_map_count setting. Which basically blocks running Elasticsearch on a GCP Autopilot cluster. Which in turn blocks running SonarQube on there, since it has Elasticsearch built in.
This Elasticsearch issue could be avoided if it would be possible to configure SonarQube to use an external Elasticsearch instance (that is running somewhere else).
It’s no yet possible to configure an external Elasticsearch.
This is not in the short-term roadmap but we are tracking the requests on this topic and are actively monitoring this thread.
Chris,
Thanks for sharing! Is it still impossible to use the external ES cluster for the community edition? Are there any roadmap documents that we can have a look at?
The K8s team on my organization doesn’t allow to run any applications with high privileges, so I still need to run Sonarqube with Docker Compose. If I could run ElasticSearch outside, this wouldn’t be a problem anymore.
We have massive performance problems as well. In the past we had a KVM based setup and now in our managed K8S cluster, we may only choose between CEPH or host-directory for block storage.
With CEPH ElasticSearch slows down analysis of bigger projects (300k LOC) by factor 3 (i.e. up to 6 minutes instead of 1.5). Other projects are slower as well, but people do not care (much) if the analysis takes 24 seconds instead of 12.
With host-directory a complete reindexing happens whenever the node is drained.
Letting us configure external ES would help here.
We have a 10M LOC Enterprise edition, of course we could complicate our setup more and possibly buy data-center, but in the moment I tend to look into moving SQ back to KVM, which is unfortunate for other reasons of course.
Being able to use cloud hosted ES would make SQ way cheaper and simpler to operate. Can’t just go around fiddling with kernel parameters on EC2 instances.