SonarQube doesn't start when using Helm with plugins

gian1200 · May 23, 2024, 10:54pm

SonarQube version

Community Edition v9.9.5 (build 90363)

How is SonarQube deployed:

Helm

What are you trying to achieve

Deploy SonarQube to a cloud Kubernetes instance WITH predefined plugins in the values.yaml file.

What have you tried so far to achieve this

Deployment without plugins, works flawlessly.
Also, manual installation of plugins (with Marketplace), works. I understand that this means that persistence is correctly configured.

When Helm is used with 1 or more plugins, I get the following error when pod starts (helm command completes without issues)

Back-off restarting failed container install-plugins in pod sonarqube-sonarqube-0_sonarqube

I also tried

values.yaml file (edited):

# Default values for sonarqube.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

# If the deployment Type is set to Deployment sonarqube is deployed as a replica set.
deploymentType: "StatefulSet"

# There should not be more than 1 sonarqube instance connected to the same database. Please set this value to 1 or 0 (in case you need to scale down programmatically).
replicaCount: 1

# How many revisions to retain (Deployment ReplicaSets or StatefulSets)
revisionHistoryLimit: 10

# This will use the default deployment strategy unless it is overriden
deploymentStrategy: {}
# Uncomment this to scheduler pods on priority
# priorityClassName: "high-priority"

## Use an alternate scheduler, e.g. "stork".
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
# schedulerName:

## Is this deployment for OpenShift? If so, we help with SCCs
OpenShift:
  enabled: false
  createSCC: true

edition: "community"

image:
  repository: sonarqube
  tag: lts-{{ .Values.edition }}
  pullPolicy: IfNotPresent
  # If using a private repository, the imagePullSecrets to use
  # pullSecrets:
  #   - name: my-repo-secret

# Set security context for sonarqube pod
securityContext:
  fsGroup: 0

# Set security context for sonarqube container
containerSecurityContext:
  # Sonarqube dockerfile creates sonarqube user as UID and GID 0
  # Those default are used to match pod security standard restricted as least privileged approach
  allowPrivilegeEscalation: false
  runAsNonRoot: true
  runAsUser: 1000
  runAsGroup: 0
  seccompProfile:
    type: RuntimeDefault
  capabilities:
    drop: ["ALL"]

# Settings to configure elasticsearch host requirements
elasticsearch:
  # DEPRECATED: Use initSysctl.enabled instead
  configureNode: false
  bootstrapChecks: true

service:
  type: ClusterIP
  externalPort: 9000
  internalPort: 9000
  labels:
  annotations: {}
  # May be used in example for internal load balancing in GCP:
  # cloud.google.com/load-balancer-type: Internal
  # loadBalancerSourceRanges:
  #   - 0.0.0.0/0
  # loadBalancerIP: 1.2.3.4

# Optionally create Network Policies
networkPolicy:
  enabled: false

  # If you plan on using the jmx exporter, you need to define where the traffic is coming from
  prometheusNamespace: "monitoring"

  # If you are using a external database and enable network Policies to be created
  # you will need to explicitly allow egress traffic to your database
  # expects https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#networkpolicyspec-v1-networking-k8s-io
  # additionalNetworkPolicys:

# will be used as default for ingress path and probes path, will be injected in .Values.env as SONAR_WEB_CONTEXT
# if .Values.env.SONAR_WEB_CONTEXT is set, this value will be ignored
sonarWebContext: ""

# (DEPRECATED) please use ingress-nginx instead
# nginx:
#   enabled: false

# Install the nginx ingress helm chart
ingress-nginx:
  enabled: false

  # You can add here any values from the official nginx ingress chart
  # controller:
  #   replicaCount: 3

ingress:
  enabled: false
  # Used to create an Ingress record.
  hosts:
    - name: sonarqube.your-org.com
      # Different clouds or configurations might need /* as the default path
      # path: /
      # For additional control over serviceName and servicePort
      # serviceName: someService
      # servicePort: somePort
      # the pathType can be one of the following values: Exact|Prefix|ImplementationSpecific(default)
      # pathType: ImplementationSpecific
  annotations: {}
  # kubernetes.io/tls-acme: "true"

  # Set the ingressClassName on the ingress record
  # ingressClassName: nginx

  # Additional labels for Ingress manifest file
  # labels:
  #  traffic-type: external
  #  traffic-type: internal
  tls: []
  # Secrets must be manually created in the namespace. To generate a self-signed certificate (and private key) and then create the secret in the cluster please refer to official documentation available at https://kubernetes.github.io/ingress-nginx/user-guide/tls/#tls-secrets
  # - secretName: chart-example-tls
  #   hosts:
  #     - chart-example.local

route:
  enabled: false
  host: ""
  # Add tls section to secure traffic. TODO: extend this section with other secure route settings
  # Comment this out if you want plain http route created.
  tls:
    termination: edge

  annotations: {}
  # See Openshift/OKD route annotation
  # https://docs.openshift.com/container-platform/4.10/networking/routes/route-configuration.html#nw-route-specific-annotations_route-configuration
  # haproxy.router.openshift.io/timeout: 1m

  # Additional labels for Route manifest file
  # labels:
  #  external: 'true'

# Affinity for pod assignment
# Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}

# Tolerations for pod assignment
# Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
# taint a node with the following command to mark it as not schedulable for new pods
# kubectl taint nodes <node> sonarqube=true:NoSchedule
# The following statement will tolerate this taint and as such reverse a node for sonarqube
tolerations: []
#  - key: "sonarqube"
#    operator: "Equal"
#    value: "true"
#    effect: "NoSchedule"

# Node labels for pod assignment
# Ref: https://kubernetes.io/docs/user-guide/node-selection/
# add a label to a node with the following command
# kubectl label node <node> sonarqube=true
nodeSelector: {}
#  sonarqube: "true"

# hostAliases allows the modification of the hosts file inside a container
hostAliases: []
# - ip: "192.168.1.10"
#   hostnames:
#   - "example.com"
#   - "www.example.com"

readinessProbe:
  initialDelaySeconds: 120
  periodSeconds: 30
  failureThreshold: 6
  # Note that timeoutSeconds was not respected before Kubernetes 1.20 for exec probes
  timeoutSeconds: 1
  # If an ingress *path* other than the root (/) is defined, it should be reflected here
  # A trailing "/" must be included
  # deprecated please use sonarWebContext at the value top level
  # sonarWebContext: /

livenessProbe:
  initialDelaySeconds: 480
  periodSeconds: 30
  failureThreshold: 6
  # Note that timeoutSeconds was not respected before Kubernetes 1.20 for exec probes
  timeoutSeconds: 1
  # If an ingress *path* other than the root (/) is defined, it should be reflected here
  # A trailing "/" must be included
  # deprecated please use sonarWebContext at the value top level
  # sonarWebContext: /

startupProbe:
  initialDelaySeconds: 240
  periodSeconds: 10
  failureThreshold: 24
  # Note that timeoutSeconds was not respected before Kubernetes 1.20 for exec probes
  timeoutSeconds: 1
  # If an ingress *path* other than the root (/) is defined, it should be reflected here
  # A trailing "/" must be included
  # deprecated please use sonarWebContext at the value top level
  # sonarWebContext: /

initContainers:
  # image: busybox:1.36
  # We allow the init containers to have a separate security context declaration because
  # the initContainer may not require the same as SonarQube.
  # Those default are used to match pod security standard restricted as least privileged approach
  securityContext:
    allowPrivilegeEscalation: false
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 0
    seccompProfile:
      type: RuntimeDefault
    capabilities:
      drop: ["ALL"]
  # We allow the init containers to have a separate resources declaration because
  # the initContainer does not take as much resources.
  resources: {}

# Extra init containers to e.g. download required artifacts
extraInitContainers: {}

## Array of extra containers to run alongside the sonarqube container
##
## Example:
## - name: myapp-container
##   image: busybox
##   command: ['sh', '-c', 'echo Hello && sleep 3600']
##
extraContainers: []

## Provide a secret containing one or more certificate files in the keys that will be added to cacerts
## The cacerts file will be set via SONARQUBE_WEB_JVM_OPTS and SONAR_CE_JAVAOPTS
##
caCerts:
  enabled: false
  image: adoptopenjdk/openjdk11:alpine
  secret: <redacted>

initSysctl:
  enabled: true
  vmMaxMapCount: 524288
  fsFileMax: 131072
  nofile: 131072
  nproc: 8192
  # image: busybox:1.36
  securityContext:
    # Compatible with podSecurity standard privileged
    privileged: true
    # if run without root permissions, error "sysctl: permission denied on key xxx, ignoring"
    runAsUser: 0
  # resources: {}

# This should not be required anymore, used to chown/chmod folder created by faulty CSI driver that are not applying properly POSIX fsgroup.
initFs:
  enabled: false
  # Image: busybox:1.36
  # Compatible with podSecurity standard baseline.
  securityContext:
    privileged: false
    runAsNonRoot: false
    runAsUser: 0
    runAsGroup: 0
    seccompProfile:
      type: RuntimeDefault
    capabilities:
      drop: ["ALL"]
      add: ["CHOWN"]

prometheusExporter:
  enabled: false
  # jmx_prometheus_javaagent version to download from Maven Central
  version: "0.17.2"
  # Alternative full download URL for the jmx_prometheus_javaagent.jar (overrides prometheusExporter.version)
  # downloadURL: ""
  # if you need to ignore TLS certificates for whatever reason enable the following flag
  noCheckCertificate: false

  # Ports for the jmx prometheus agent to export metrics at
  webBeanPort: 8000
  ceBeanPort: 8001

  config:
    rules:
      - pattern: ".*"
  # Overrides config for the CE process Prometheus exporter (by default, the same rules are used for both the Web and CE processes).
  # ceConfig:
  #   rules:
  #     - pattern: ".*"
  # image: curlimages/curl:8.2.1
  # For use behind a corporate proxy when downloading prometheus
  # httpProxy: ""
  # httpsProxy: ""
  # noProxy: ""
  # Reuse default initcontainers.securityContext that match restricted pod security standard
  # securityContext: {}

prometheusMonitoring:
  # Generate a Prometheus Pod Monitor (https://github.com/coreos/prometheus-operator)
  #
  podMonitor:
    # Create PodMonitor Resource for Prometheus scraping
    enabled: false
    # (DEPRECATED) Specify a custom namespace where the PodMonitor will be created.
    # This value should not be set, as the PodMonitor's namespace has to match the Release Namespace.
    # namespace: "default"
    # Specify the interval how often metrics should be scraped
    interval: 30s
    # Specify the timeout after a scrape is ended
    # scrapeTimeout: ""
    # Name of the label on target services that prometheus uses as job name
    # jobLabel: ""

# List of plugins to install.
# For example:
# plugins:
#  install:
#    - "https://github.com/AmadeusITGroup/sonar-stash/releases/download/1.3.0/sonar-stash-plugin-1.3.0.jar"
#    - "https://github.com/SonarSource/sonar-ldap/releases/download/2.2-RC3/sonar-ldap-plugin-2.2.0.601.jar"
#
plugins:
  install:
    - "https://github.com/dependency-check/dependency-check-sonar-plugin/releases/download/4.0.1/sonar-dependency-check-plugin-4.0.1.jar"

  # For use behind a corporate proxy when downloading plugins
  # httpProxy: ""
  # httpsProxy: ""
  # noProxy: ""

  # image: curlimages/curl:8.2.1
  # resources: {}

  # .netrc secret file with a key "netrc" to use basic auth while downloading plugins
  # netrcCreds: ""

  # Set to true to not validate the server's certificate to download plugin
  noCheckCertificate: true
  # Reuse default initcontainers.securityContext that match restricted pod security standard
  # securityContext: {}

## (DEPRECATED) The following value sets SONAR_WEB_JAVAOPTS (e.g., jvmOpts: "-Djava.net.preferIPv4Stack=true"). However, this is deprecated, please set SONAR_WEB_JAVAOPTS or sonar.web.javaOpts directly instead.
jvmOpts: ""

## (DEPRECATED) The following value sets SONAR_CE_JAVAOPTS. However, this is deprecated, please set SONAR_CE_JAVAOPTS or sonar.ce.javaOpts directly instead.
jvmCeOpts: ""

## a monitoring passcode needs to be defined in order to get reasonable probe results
# not setting the monitoring passcode will result in a deployment that will never be ready
monitoringPasscode: "<redacted>"
# Alternatively, you can define the passcode loading it from an existing secret specifying the right key
# monitoringPasscodeSecretName: "pass-secret-name"
# monitoringPasscodeSecretKey: "pass-key"

## Environment variables to attach to the pods
##
# env:
#   # If you use a different ingress path from /, you have to add it here as the value of SONAR_WEB_CONTEXT
#   - name: SONAR_WEB_CONTEXT
#     value: /sonarqube
#   - name: VARIABLE
#     value: my-value

# Set annotations for pods
annotations: {}

## We usually don't make specific resource recommendations, as they are heavily dependend on
## the usage of SonarQube and the surrounding infrastructure.
## Those default are based on the default Web -Xmx1G -Xms128m and CE -Xmx2G -Xms128m and Search -Xmx2G -Xms2G settings of SQ sub processes
## Adjust these values to your needs, you can find more details on the main README of the chart.
resources:
  limits:
    cpu: 800m
    memory: 2G
    ephemeral-storage: 5G
  requests:
    cpu: 100m
    memory: 1536M
    ephemeral-storage: 1536M

persistence:
  enabled: true
  ## Set annotations on pvc
  annotations: {}

  ## Specify an existing volume claim instead of creating a new one.
  ## When using this option all following options like storageClass, accessMode and size are ignored.
  # existingClaim:

  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  storageClass: <redacted>
  accessMode: ReadWriteOnce
  size: 5Gi
  uid: 1000
  guid: 0

  ## Specify extra volumes. Refer to ".spec.volumes" specification : https://kubernetes.io/fr/docs/concepts/storage/volumes/
  volumes: []
  ## Specify extra mounts. Refer to ".spec.containers.volumeMounts" specification : https://kubernetes.io/fr/docs/concepts/storage/volumes/
  mounts: []

# In case you want to specify different resources for emptyDir than {}
emptyDir:
  {}
  # Example of resouces that might be used:
  # medium: Memory
  # sizeLimit: 16Mi

# A custom sonar.properties file can be provided via dictionary.
# For example:
# sonarProperties:
#   sonar.forceAuthentication: true
#   sonar.security.realm: LDAP
#   ldap.url: ldaps://organization.com

# Additional sonar properties to load from a secret with a key "secret.properties" (must be a string)
# sonarSecretProperties:

# Kubernetes secret that contains the encryption key for the sonarqube instance.
# The secret must contain the key 'sonar-secret.txt'.
# The 'sonar.secretKeyPath' property will be set automatically.
# sonarSecretKey: "settings-encryption-secret"

## Override JDBC values
## for external Databases
jdbcOverwrite:
  # If enable the JDBC Overwrite, make sure to set `postgresql.enabled=false`
  enable: true
  # The JDBC url of the external DB
  jdbcUrl: "jdbc:postgresql://<redacted>/sonarqube?socketTimeout=1500"
  # The DB user that should be used for the JDBC connection
  jdbcUsername: "<redacted>"
  # Use this if you don't mind the DB password getting stored in plain text within the values file
  # jdbcPassword: ""
  ## Alternatively, use a pre-existing k8s secret containing the DB password
  jdbcSecretName: "<redacted>"
  ## and the secretValueKey of the password found within that secret
  jdbcSecretPasswordKey: "<redacted>"

## (DEPRECATED) Configuration values for postgresql dependency
## ref: https://github.com/bitnami/charts/blob/master/bitnami/postgresql/README.md
postgresql:
  # Enable to deploy the bitnami PostgreSQL chart
  enabled: false
  ## postgresql Chart global settings
  # global:
  #   imageRegistry: ''
  #   imagePullSecrets: ''
  ## bitnami/postgres image tag
  # image:
  #   tag: 11.7.0-debian-10-r9
  # existingSecret Name of existing secret to use for PostgreSQL passwords
  # The secret has to contain the keys postgresql-password which is the password for postgresqlUsername when it is
  # different of postgres, postgresql-postgres-password which will override postgresqlPassword,
  # postgresql-replication-password which will override replication.password and postgresql-ldap-password which will be
  # used to authenticate on LDAP. The value is evaluated as a template.
  # existingSecret: ""
  #
  # The bitnami chart enforces the key to be "postgresql-password". This value is only here for historic purposes
  # existingSecretPasswordKey: "postgresql-password"
  postgresqlUsername: "<redacted>"
  postgresqlPassword: "<redacted>"
  postgresqlDatabase: "<redacted>"
  # Specify the TCP port that PostgreSQL should use
  service:
    port: 5432
  resources:
    limits:
      cpu: 2
      memory: 2Gi
    requests:
      cpu: 100m
      memory: 200Mi
  persistence:
    enabled: true
    accessMode: ReadWriteOnce
    size: 20Gi
    storageClass:
  securityContext:
    # For standard Kubernetes deployment, set enabled=true
    # If using OpenShift, enabled=false for restricted SCC and enabled=true for anyuid/nonroot SCC
    enabled: true
    # fsGroup specification below are not applied if enabled=false. enabled=false is the required setting for OpenShift "restricted SCC" to work successfully.
    # postgresql dockerfile sets user as 1001
    fsGroup: 1001
  containerSecurityContext:
    # For standard Kubernetes deployment, set enabled=true
    # If using OpenShift, enabled=false for restricted SCC and enabled=true for anyuid/nonroot SCC
    enabled: true
    # runAsUser specification below are not applied if enabled=false. enabled=false is the required setting for OpenShift "restricted SCC" to work successfully.
    # postgresql dockerfile sets user as 1001, the rest aim at making it compatible with restricted pod security standard.
    runAsUser: 1001
    allowPrivilegeEscalation: false
    runAsNonRoot: true
    seccompProfile:
      type: RuntimeDefault
    capabilities:
      drop: ["ALL"]
  volumePermissions:
    # For standard Kubernetes deployment, set enabled=false
    # For OpenShift, set enabled=true and ensure to set volumepermissions.securitycontext.runAsUser below.
    enabled: false
    # if using restricted SCC set runAsUser: "auto" and if running under anyuid/nonroot SCC - runAsUser needs to match runAsUser above
    securityContext:
      runAsUser: 0
  shmVolume:
    chmod:
      enabled: false
  serviceAccount:
    ## If enabled = true, and name is not set, postgreSQL will create a serviceAccount
    enabled: false
    # name:

# Additional labels to add to the pods:
# podLabels:
#   key: value
podLabels: {}
# For compatibility with 8.0 replace by "/opt/sq"
# For compatibility with 8.2, leave the default. They changed it back to /opt/sonarqube
sonarqubeFolder: /opt/sonarqube

tests:
  image: ""
  enabled: true
  resources:
    limits:
      cpu: 500m
      memory: 200M

# For OpenShift set create=true to ensure service account is created.
serviceAccount:
  create: false
  # name:
  # automountToken: false # default
  ## Annotations for the Service Account
  annotations: {}

# extraConfig is used to load Environment Variables from Secrets and ConfigMaps
# which may have been written by other tools, such as external orchestrators.
#
# These Secrets/ConfigMaps are expected to contain Key/Value pairs, such as:
#
# apiVersion: v1
# kind: ConfigMap
# metadata:
#   name: external-sonarqube-opts
# data:
#   SONARQUBE_JDBC_USERNAME: foo
#   SONARQUBE_JDBC_URL: jdbc:postgresql://db.example.com:5432/sonar
#
# These vars can then be injected into the environment by uncommenting the following:
#
# extraConfig:
#   configmaps:
#     - external-sonarqube-opts

extraConfig:
  secrets: []
  configmaps: []

# account:
# The values can be set to define the current and the (new) custom admin passwords at the startup (the username will remain "admin")
#   adminPassword: admin
#   currentAdminPassword: admin
# The above values can be also provided by a secret that contains "password" and "currentPassword" as keys. You can generate such a secret in your cluster
# using "kubectl create secret generic admin-password-secret-name --from-literal=password=admin --from-literal=currentPassword=admin"
#   adminPasswordSecretName: ""
# # Reuse default initcontainers.securityContext that match restricted pod security standard
# #   securityContext: {}
#   resources:
#     limits:
#       cpu: 100m
#       memory: 128Mi
#     requests:
#       cpu: 100m
#       memory: 128Mi
# curlContainerImage: curlimages/curl:8.2.1
# adminJobAnnotations: {}
# deprecated please use sonarWebContext at the value top level
#   sonarWebContext: /

terminationGracePeriodSeconds: 60

Colin · May 27, 2024, 6:39am

Hey there!

What version of the helm chart are you using?

gian1200 · May 27, 2024, 8:20am

Hi, I think it’s last version. It was freshly downloaded last week

Not sure how to confirm that. However, the command I use to run is:

helm upgrade --install -f ./values.yml -n sonarqube sonarqube sonarqube/sonarqube

…

Now that you talk about it (and after re-reading the docs), I should add --version '~8', correct?

Colin · May 27, 2024, 12:03pm

I think that’s a good start.

gian1200 · May 27, 2024, 2:51pm

Yeah, it worked! Thanks for the support! Not sure how I missed that!

Although, it’d be nice to have it aligned with the SonarQube version that it works with.

For anyone interested, the final command was:

helm upgrade --install -f values.yaml -n sonarqube --version '~8' sonarqube sonarqube/sonarqube

Colin · May 27, 2024, 2:54pm

Glad it helped! To be honest… I have no idea what specifically changed (it was just a guess)

That’s the case now starting from v10.0

[10.0.0]

Update SonarQube to 10.0.0

Helm chart versioning will now follow the SonarQube product versioning

Cheers!

system · June 3, 2024, 2:55pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.