Sonarqube Helm chart deployment (enterprise edition)

Sooter_Saalu · August 17, 2024, 4:00pm

Must-share information (formatted with Markdown):

which versions are you using - latest charts (10.6.0 according to the github)(SonarQube, Scanner, Plugin, and any relevant extension)
how is SonarQube deployed: Helm
what are you trying to achieve - my goal is to install sonarqube enterprise edition on an aws cluster using helm
what have you tried so far to achieve this. I have used the helm chart connecting to an rds database. i get perpetually pending pods when persistence is enabled. also tried to deploy the chart as is and that also gets a pending pod for the PostgreSQL which I think holds up everything else

Do not share screenshots of logs – share the text itself (bonus points for being well-formatted)!

running kubectl logs sonarqube-sonarqube-0 -n sonarqube gives me
“Defaulted container “sonarqube” out of: sonarqube, init-sysctl (init)”
running kubectl logs sonarqube-sonarqube-0 init-sysctl -n sonarqube gives me nothing

Sooter_Saalu · August 17, 2024, 4:42pm

Sonarqube Helm chart for community support · GitHub This helm chart(persistence disabled) works i.e the pod is running and ready, but i can’t reach the web server on the pod and instance IP. Ports are open at 9000

jeremy.cotineau · August 23, 2024, 8:17am

Hello @Sooter_Saalu thanks a lot for participating in the community.

Could I ask you if you’ve used any ingress/ingress controllers? what access method are you using to access the app?

Sooter_Saalu · August 23, 2024, 8:47am

Thank you for responding. I didn’t set up an ingress. Tried port-forwarding and using the pods public ip

jeremy.cotineau · August 27, 2024, 11:59am

If the pods are healthy and up and running, port forwarding and access on localhost:9000 should work. Please make sure you don’t have SonarWebContext set; otherwise, you need to suffix that to the URL.

the persistency is optional and slightly improves performances, if you cluster does not have proper storage class this might be why the pods keeps being in the pending state.

Could you share a bit more info, like kubectl get pods and the error you get when accessing through port forwarding?

Sooter_Saalu · August 29, 2024, 11:40am

thank you. there’s the default gp2. should I create another one. what would be the required/proper settings?

“Name: gp2
IsDefaultClass: Yes
Annotations: kubectl.kubernetes.io/last-applied-configuration={“apiVersion”:“storage.k8s.io/v1”,“kind”:“StorageClass”,“metadata”:{“annotations”:{“storageclass.kubernetes.io/is-default-class”:“true”},“name”:“gp2”},“parameters”:{“fsType”:“ext4”,“type”:“gp2”},“provisioner”:“kubernetes.io/aws-ebs",“volumeBindingMode”:"WaitForFirstConsumer”}
,storageclass.kubernetes.io/is-default-class=true
Provisioner: kubernetes.io/aws-ebs
Parameters: fsType=ext4,type=gp2
AllowVolumeExpansion:
MountOptions:
ReclaimPolicy: Delete
VolumeBindingMode: WaitForFirstConsumer
Events: ”

Web context is left empty. kubectl get pods -n sonarqube shows the pod running. added a screenshot. The error when accessing the pod IP is “cant reach, took too long to respond”. security group rules are open to 9000.

jeremy.cotineau · August 29, 2024, 12:22pm

Your storage class is marked as default; nothing else should be required; we will troubleshoot this afterward.

With the Pod IP it makes sense that you cannot access it as it will not be exposed outside of the cluster unless you specify NodePort or LoadBalancer in the chart service type, which is not recommended for production as it would expose directly sonarqube without a reverse proxy, but this might help for debugging.

Else, I see the port-forwarding seems to work, at least it does not close automatically.

What happens when you curl localhost:8080 from the exact same machine/cloudShell you are running the port-forward?

Else you can try the full loop and install an ingress controller plus activate SonarQube helm chart ingress.

Is this a resource you are familiar with?

Sooter_Saalu · August 29, 2024, 5:17pm

curling 8080 from the cloudshell goes through.

<body>
    <div id="content" data-base-url="" data-server-status="UP" data-instance="SonarQube" data-official="true">
        <div class="global-loading">
            <i class="global-loading-spinner"></i>
            <span aria-live="polite" class="global-loading-text">Loading...</span>
        </div>

for the ingress - so first deploying the “ingress-nginx” controller then editing the sonarqube values.yaml

ingress-nginx: 
 enabled: true

ingress:
 enabled: true
 hosts:
   name:# can I use an associated Public IPv4 DNS?
... 
annotations

jeremy.cotineau · August 30, 2024, 7:03am

nice to hear that!

If you are not familiar with ingresses, I suggest you extensively read the concept and documentation; this is the critical software that will expose your application and can cause many security issues.

The workflow is indeed to deploy the ingress-nginx, which will create a loadBalancer kubernetes service. You’ll then have to properly configure your SonarQube DNS name to point to this loadbalancer IP, and then create the ingress and put the DNS name in the host section of the ingress.

Sooter_Saalu · September 3, 2024, 5:47pm

@jeremy.cotineau Thank you very much for that.
Do you have any ideas on debugging the persistence

jeremy.cotineau · September 4, 2024, 7:01am

Sure, you can activate it back, and when the pod will be pending, there are multiple commands to see what happens.

kubectl get events
kubectl describe pod/deployment

Those two commands with the name of the concerned object, should tell you why the pod is pending. If you do not have anything here, then you should take a look at the logs of your CSI controller.

Sooter_Saalu · September 4, 2024, 7:53am

@jeremy.cotineau
describe

Name:             sonarqube-sonarqube-0
Namespace:        sonarqube
Priority:         0
Service Account:  default
Node:             <none>
Labels:           app=sonarqube
                  apps.kubernetes.io/pod-index=0
                  controller-revision-hash=sonarqube-sonarqube-6678fd658b
                  release=sonarqube
                  statefulset.kubernetes.io/pod-name=sonarqube-sonarqube-0
Annotations:      checksum/config: 5544f38762b6fef34865aa34057b29c4288cb68ea50ed7dea0e00d2bae2a147d
                  checksum/init-fs: 666df574d12897dd258754e9dc0f8be60e51a5dae582657f398cd1d21dfc0967
                  checksum/init-sysctl: f368552ed051397020914eb3dfbb8a13fa2745e307f7e1f3665ddbdd267264a1
                  checksum/plugins: c5a8e5568c6a915d43f0e5bd0307360d3b146896677b5c94f94762abda803748
                  checksum/secret: 870e91703075b4f93cff434e8a264e1709f6dd7a8e095a5bb55811c96cbb5fd9
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    StatefulSet/sonarqube-sonarqube
Init Containers:
  init-sysctl:
    Image:      sonarqube:10.6.0-enterprise
    Port:       <none>
    Host Port:  <none>
    Command:
      /bin/bash
      -e
      /tmp/scripts/init_sysctl.sh
    Environment:
      SONAR_WEB_CONTEXT:   /
      SONAR_WEB_JAVAOPTS:  
      SONAR_CE_JAVAOPTS:   
    Mounts:
      /tmp/scripts/ from init-sysctl (rw)
Containers:
  sonarqube:
    Image:           sonarqube:10.6.0-enterprise
    Port:            9000/TCP
    Host Port:       0/TCP
    SeccompProfile:  RuntimeDefault
    Limits:
      cpu:                800m
      ephemeral-storage:  512G
      memory:             6G
    Requests:
      cpu:                400m
      ephemeral-storage:  5G
      memory:             5G
    Liveness:             exec [sh -c wget --no-proxy --quiet -O /dev/null --timeout=1 --header="X-Sonar-Passcode: $SONAR_WEB_SYSTEMPASSCODE" "http://localhost:9000/api/system/liveness"
] delay=60s timeout=1s period=30s #success=1 #failure=6
    Readiness:  exec [sh -c #!/bin/bash
# A Sonarqube container is considered ready if the status is UP, DB_MIGRATION_NEEDED or DB_MIGRATION_RUNNING
# status about migration are added to prevent the node to be kill while sonarqube is upgrading the database.
if wget --no-proxy -qO- http://localhost:9000/api/system/status | grep -q -e '"status":"UP"' -e '"status":"DB_MIGRATION_NEEDED"' -e '"status":"DB_MIGRATION_RUNNING"'; then
  exit 0
fi
exit 1
] delay=60s timeout=1s period=30s #success=1 #failure=6
    Startup:  http-get http://:http/api/system/status delay=30s timeout=1s period=10s #success=1 #failure=24
    Environment Variables from:
      sonarqube-sonarqube-jdbc-config  ConfigMap  Optional: false
    Environment:
      SONAR_HELM_CHART_VERSION:  10.6.1_3163
      SONAR_JDBC_PASSWORD:       <set to the key 'jdbc-password' in secret 'sonarqube-sonarqube'>                                 Optional: false
      SONAR_WEB_SYSTEMPASSCODE:  <set to the key 'SONAR_WEB_SYSTEMPASSCODE' in secret 'sonarqube-sonarqube-monitoring-passcode'>  Optional: false
      SONAR_WEB_CONTEXT:         /
      SONAR_WEB_JAVAOPTS:        
      SONAR_CE_JAVAOPTS:         
    Mounts:
      /opt/sonarqube/data from sonarqube (rw,path="data")
      /opt/sonarqube/extensions from sonarqube (rw,path="extensions")
      /opt/sonarqube/logs from sonarqube (rw,path="logs")
      /opt/sonarqube/temp from sonarqube (rw,path="temp")
      /tmp from tmp-dir (rw)
Volumes:
  init-sysctl:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      sonarqube-sonarqube-init-sysctl
    Optional:  false
  sonarqube:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  sonarqube-sonarqube
    ReadOnly:   false
  tmp-dir:
    Type:        EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:      
    SizeLimit:   <unset>
QoS Class:       Burstable
Node-Selectors:  sonarqube=true
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
                 sonarqube=true:NoSchedule
Events:          <none>

kubectl events --for pod/sonarqube-sonarqube-0 -n sonarqube

LAST SEEN           TYPE      REASON             OBJECT                      MESSAGE
35m                 Normal    Scheduled          Pod/sonarqube-sonarqube-0   Successfully assigned sonarqube/sonarqube-sonarqube-0 to ip-172-31-89-17.ec2.internal
35m                 Normal    Pulling            Pod/sonarqube-sonarqube-0   Pulling image "sonarqube:10.6.0-enterprise"
34m                 Normal    Pulled             Pod/sonarqube-sonarqube-0   Successfully pulled image "sonarqube:10.6.0-enterprise" in 53.696s (53.696s including waiting)
34m                 Normal    Created            Pod/sonarqube-sonarqube-0   Created container init-sysctl
34m                 Normal    Started            Pod/sonarqube-sonarqube-0   Started container init-sysctl
33m                 Normal    Pulled             Pod/sonarqube-sonarqube-0   Container image "sonarqube:10.6.0-enterprise" already present on machine
33m                 Normal    Created            Pod/sonarqube-sonarqube-0   Created container sonarqube
33m                 Normal    Started            Pod/sonarqube-sonarqube-0   Started container sonarqube
33m                 Warning   Unhealthy          Pod/sonarqube-sonarqube-0   Startup probe failed: Get "http://172.31.84.135:9000/api/system/status": dial tcp 172.31.84.135:9000: connect: connection refused
32m (x2 over 33m)   Warning   Unhealthy          Pod/sonarqube-sonarqube-0   Startup probe failed: Get "http://172.31.84.135:9000/api/system/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
30m (x7 over 32m)   Warning   Unhealthy          Pod/sonarqube-sonarqube-0   Readiness probe failed:
30m                 Warning   Unhealthy          Pod/sonarqube-sonarqube-0   Liveness probe failed:
12m                 Normal    Killing            Pod/sonarqube-sonarqube-0   Stopping container sonarqube
7m5s                Warning   FailedScheduling   Pod/sonarqube-sonarqube-0   running PreBind plugin "VolumeBinding": binding volumes: pod does not exist any more: pod "sonarqube-sonarqube-0" not found

Sooter_Saalu · September 9, 2024, 3:52am

Looks like it was permission based. it used ebs as an external provisioner and an IAM role is needed with aws

Thank you!

system · September 16, 2024, 3:52am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Deploying sonarqube using helm chart provided by sonarqube SonarQube Server / Community Build	15	1181	July 30, 2024
LTS deployed via helm; Stuck on "Loading..." SonarQube Server / Community Build kubernetes , helm , aws	6	1035	November 13, 2023
Best practice when deploying the helm chart of sonarqube? Discussions sonarqube , kubernetes , helm	3	217	July 15, 2024
Deploying sonarqube in the cloud with an external ingress SonarQube Server / Community Build sonarqube , kubernetes , helm , argo	7	3358	March 8, 2022
Using RDS/MySQL as database for deployment in Kubernetes SonarQube Server / Community Build	1	385	January 25, 2024

Sonarqube Helm chart deployment (enterprise edition)

Related topics