SonarQube, Fargate & AWS ElasticSearch

SonarQube
1

Hi, I’m running SonarQube 7.9 LTS Community. I am hosting it on AWS using Fargate, RDS and it’s working. I’d like to however switch from using the inbuilt SonarQube ElasticSearch to the AWS ElasticSearch.

Is this possible when using the Community Edition? I ask because essentially what I am trying to setup is pretty much the DataCenter edition as described here https://docs.sonarqube.org/latest/setup/install-cluster/. It states " Running SonarQube as a Cluster is only possible with a Data Center Edition ."

I am not really looking for high availability. It just isn’t good practice to run ElasticSearch within a Fargate container as the Elasticsearch indexes will be deleted whenever the container is replaced.

In addtion to the above the SonarQube doco states:
" By default, Elasticsearch data is stored in $SONARQUBE-HOME/data , but this is not recommended for production instances. Instead, you should store this data elsewhere, ideally in a dedicated volume with fast I/O. Beyond maintaining acceptable performance, doing so will also ease the upgrade of SonarQube." (https://docs.sonarqube.org/latest/setup/install-server/)

So yeah, I’d like to be able to use the AWS Elasticsearch, not the SonarQube integrated one. Is this possible?

2

Hi,

Switching to an external Elasticsearch instance isn’t possible in any edition, not even the Data Center Edition.

 
Ann

3

The documentation here https://docs.sonarqube.org/latest/setup/install-cluster/ seems to clearly indicate otherwise Ann? Obviously this documentation is for DataCenter edition.

4

Hi,

That documentation page is indeed for the Data Center Edition, which runs Elasticsearch inside SonarQube nodes.

 
Ann

5

Wow, wow… wow.

So what are the implications of running SonarQube (any edition) in a container setup where the ElasticSearch indexes are wiped anytime the container is refreshed?

6

Hi,

It means that starting back up takes longer because ES has to reindex. Really, not what we recommend. This would be one reason you’d configure the ES indices to be stored somewhere else, as mentioned in the docs you cited in your OP.

 
Ann

7

Appreciate it’s not what SonarQube recommends, but aside from a slower startup there is no material impact? If so, this should work for us in the shorter term.

In the longer term and for a more permanent solution it means we can’t host this on Fargate as it doesn’t support persistent storage :frowning:

9

Sounds like someone in the community needs to implement external Elastic Search.
Looks like Data Center edition is already able to be configured to have a “Search Node” which talks to the “application nodes”.

Or possibly setting these options…

sonar.cluster.enabled=true
sonar.cluster.hosts=ip1,ip2
sonar.cluster.search.hosts=ip3,ip4,ip5
sonar.cluster.node.type=application
sonar.auth.jwtBase64Hs256Secret=YOURGENERATEDSECRET

And then creating your own “search node”

10

Hi James,

Good to hear that you said you have setup Sonarqube on fargate and will be using RDS , Even I am looking for a similar solution , I have even setup on fargate , but I need details regarding how to connect that to RDS .

Can you please help me in that regarding the docker file for Sonarqube to map it to RDS to store the Data

Any help on this would be highly appreciated.

11

@vinodkumar4b9, you can configure the database connection string in sonar.properties file. For example, I have a line like sonar.jdbc.url=jdbc:postgresql://#POSTGRES_URL#:#POSTGRES_PORT#/#POSTGRES_DBNAME# and then I use the sed command to replace those values on container startup, replacing them with my RDS details.

12

Hi James,

Thanks for the details on this and will try accordingly , It would be helpful to me if you can share docker file if you don’t have any issues so that I can leverage the same for my work where it is like similar setup where I need to setup with AWS fargate with RDS.

Appreciated your help on this

13

Hello James,

Did you run sonarqube pod on ECS fargate or EKS fargate? I tried to run on EKS fargate but looks like it has limitations of not supporting privileged images. turned off the privileges but pod itself would not even come online.

Can you please help on this how you managed to run?

1 Like
14

is there a solution to host SonarQube on EKS fargate. Running into the same privileged issue where following commands are unable to execute.

if [[ "$(sysctl -n vm.max_map_count)" -lt 524288 ]]; then
      sysctl -w vm.max_map_count=524288
    fi
    if [[ "$(sysctl -n fs.file-max)" -lt 131072 ]]; then
      sysctl -w fs.file-max=131072
    fi
    if [[ "$(ulimit -n)" != "unlimited" ]]; then
      if [[ "$(ulimit -n)" -lt 131072 ]]; then
        echo "ulimit -n 131072"
        ulimit -n 131072
      fi
    fi
    if [[ "$(ulimit -u)" != "unlimited" ]]; then
      if [[ "$(ulimit -u)" -lt 8192 ]]; then
        echo "ulimit -u 8192"
        ulimit -u 8192
      fi
    fi

This is part of the community chart of sonarqube.