SonarQube and MS SQL

Xiny · August 21, 2021, 11:02am

Hi team,

I’m trying to deploy SonarQube 7.1 into a Kubernetes cluster (2 nodes) with 2 replicaset and connect into an external dataabse Azure SQL.

I have a couple of questions here:

In Kubernetes my deployment with 2 pods of sonarqube can share same Volume for data and extensions? Will be not any data corruption?
I get some error conection with the database (pods exits with code 143). It seems that pods connect correctly with the database but maybe is it some kind of known bug? i am asking just in case.
I will try to deploy the sonarqube deployment with a helm chart… i found this helm chart on git helm-chart-sonarqube/charts/sonarqube at master · SonarSource/helm-chart-sonarqube · GitHub but it seems that almost every chart deployment only supports Postgress or mysql but not mssql. Can I use the previous helm chart with azure sql?

Thanks in advance.

DefinitelyNotTobi · August 21, 2021, 2:58pm

Hello @Xiny and welcome to the community

sonarqube 7.1 is long EOL. when you are currently bootstraping a new instance, you should use the current LTS (8.9.2) or the latest release (9.0).

please note that only the Datacenter Edition of SonarQube can be clustered. every other edition is single pod only.

can you share some log message regarding this? i think it is rather the bundled ES that is causing issues here.

i think the chart can be used with MSSQL, if you overwrite the jdbc URL and use username/password authentication this should work did you already try this?

Xiny · August 22, 2021, 8:11am

Helo @DefinitelyNotTobi,

Many thanks for your answer!

So it means that in the part of the helm chart that says:

image:
  repository: sonarqube
  tag: 8.9-community

I need to change this for:

image:
  repository: sonarqube
  tag: 9.0.1-datacenter-search

I’ve already uploaded the logs in txt. Just mention here that I’ve created a MMSQL database through Azure SQL in Azure so not sure if this can be the issue here… I was reding something that creating a datatabse through Azure SQL and trying to connect the deployment to any external Azure SQL can produce some errors due to table incompatibilities.

new logs.txt (15.0 KB)

I will try to override the URL. I am not doing too much tests in Azure platform as i have very limited budget and not much experience into the platform and i don’t want to run into extra costs but definitely i will try to do that. Basically I need to set the posgres database to false and overide the url, right? something like this:


# Kubernetes secret that contains the encryption key for the sonarqube instance.
# The secret must contain the key 'sonar-secret.txt'.
# The 'sonar.secretKeyPath' property will be set automatically.
# sonarSecretKey: "settings-encryption-secret"

## JDBC Database Type; by default postgresql. To use a different Database type, adjust
jdbcDatabaseType: postgresql

## Override JDBC URL
jdbcUrlOverride: "jdbc:sqlserver://mydatabase.database.windows.net:1433;database=sonarqubesqldb;user=SONARQUBE_JDBC_USERNAME@mydatabase;password=SONARQUBE_JDBC_PASSWORD"

## Configuration values for postgresql dependency
## ref: https://github.com/kubernetes/charts/blob/master/stable/postgresql/README.md
postgresql:
  # Enable to deploy the PostgreSQL chart
  enabled: false
  # To use an external PostgreSQL instance, set enabled to false and uncomment
  # the line below:
  # postgresqlServer: ""
  # To use an external secret for the password for an external PostgreSQL
  # instance, set enabled to false and provide the name of the secret on the
  # line below:
  # existingSecret: ""
  # existingSecretPasswordKey: "postgresql-password"
  postgresqlUsername: "sonarUser"
  postgresqlPassword: "sonarPass"
  postgresqlDatabase: "sonarDB"
  # Specify the TCP port that PostgreSQL should use

here to include the external file with the username and password i have to specify the file name in " existingSecret: " isn’t it?

Regards!

DefinitelyNotTobi · August 22, 2021, 1:58pm

Xiny:

So it means that in the part of the helm chart that says:
image:
  repository: sonarqube
  tag: 8.9-community
I need to change this for:
image:
  repository: sonarqube
  tag: 9.0.1-datacenter-search

No. As stated in the documentation and the readme, you can only use this chart with the community, developer or enterprise edition. if you plan to bootstrap a datacenter edition i would recommend you to wait for the next release as there will be beta support for the datacenter edition on kubernetes.

The logs seem to be good in regard of the database, but you have incompatible plugins in your PVC that is causing sonarqube to stop early

2021.08.20 16:13:13 ERROR app[startup] Startup failed: Plugins can’t be loaded. See web logs for more information
2021.08.20 16:13:13 ERROR web[o.s.s.p.PlatformImpl] Web server startup failed: The following plugins are no longer compatible with this version of SonarQube: ‘scmgit’, ‘scmsvn’

you can remove them both from your PVC as they are embedded into sonarqube since 8.5 i think. after that i should be okay

Xiny · August 22, 2021, 2:20pm

I understand so, in case i am obligated to share volumes I need basically to create my own chart with the data center edition as I cannot use that one, isn’t it?

Also, when you say to delete those plugins, I can do that when i create the PVCs on the manifest YAML or I should do any extra task on the PVCs… Both PVC for data and extensions are created through a manifest YAML on Azure so not sure if i should modify the YAMLs files?

Do you think the URL overwrite to use external databse through that chart i showed you is it correct?

Thanks and regads,

DefinitelyNotTobi · August 22, 2021, 2:37pm

In that case you need to wait for the next release to get something that is officially supported (probably landing in September)

Regarding your pvc, you can just delete the one that you currently have with the plug-ins inside if you are deploying a new version where the plug-ins are already embedded. The pvc will be recreated during deployment.

The jdbc overwrite URL looks good when the username and password you wrote there are placeholder

Xiny · August 23, 2021, 10:39am

I was trying to test the new sonarqube image in my regular deployment.
Firstly I deleted my PVs and also my PVCs and later on re-created them. After that I re-deployed again the kuberntes cluster just with one replicaset.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: sonarqube
  name: sonarqube
spec:
  selector:
    matchLabels:
      app: sonarqube
  replicas: 1
  template:
    metadata:
      labels:
        app: sonarqube
    spec:
      containers:
        - name: sonarqube
          image: sonarqube:8.9-community
          resources:
            requests:
              cpu: 500m
              memory: 1024Mi
            limits:
              cpu: 2000m
              memory: 4096Mi
          volumeMounts:
          - mountPath: "/opt/sonarqube/data/"
            name: sonar-data-new
          - mountPath: "/opt/sonarqube/extensions/"
            name: sonar-extensions-new
          env:
          - name: "SONARQUBE_JDBC_USERNAME"
            valueFrom:
               secretKeyRef:
                 name: mssql-secrets
                 key: username
          - name: "SONARQUBE_JDBC_PASSWORD"
            valueFrom:
               secretKeyRef:
                 name: mssql-secrets
                 key: password
          - name: "SONARQUBE_JDBC_URL"
            valueFrom:
              configMapKeyRef:
                name: sonar-config
                key: url
          ports:
          - containerPort: 9000
            protocol: TCP
      volumes:
      - name: sonar-data-new
        persistentVolumeClaim:
          claimName: sonar-data-new
      - name: sonar-extensions-new
        persistentVolumeClaim:
          claimName: sonar-extensions-new

logs (2).txt (1005 Bytes)

I get the same error:

2021.08.23 10:34:40 INFO  web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /opt/sonarqube
2021.08.23 10:34:40 INFO  web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled
2021.08.23 10:34:41 ERROR app[][startup] Startup failed: Plugins can't be loaded. See web logs for more information
2021.08.23 10:34:41 ERROR web[][o.s.s.p.PlatformImpl] Web server startup failed: The following plugins are no longer compatible with this version of SonarQube: 'scmgit', 'scmsvn'
2021.08.23 10:34:41 INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2021.08.23 10:34:41 INFO  web[][o.s.p.ProcessEntryPoint] Hard stopping process
2021.08.23 10:34:41 INFO  app[][o.s.a.SchedulerImpl] Process[web] is stopped
2021.08.23 10:34:41 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 143
2021.08.23 10:34:41 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
2021.08.23 10:34:41 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

Status of the pods is:

CrashLoopBackOff 3 2m28s

DefinitelyNotTobi · August 23, 2021, 4:48pm

hmm i removed everything from your yaml that i can not reproduce (secrets, cm and pvc) and sonarqube is starting up just fine for me, so i guess your pvc is not clean

Xiny · August 23, 2021, 5:19pm

Thanks for that!

It is so wierd as I created from scratch those PVs and PVCs just for that deployment… Basically I deleted everything including the deployment, change the name of PVs and PVCs and re deploy the volums, volumes claims and the deployment.

I have a couple of questions…

Have you used Azure SQL for that test?
May you please attach how you setup your PVs and PVCs just to compare with my deployment?

Regards!

Xiny · August 24, 2021, 10:54am

logs3.txt (7.3 KB)
I tried with a fresh database, disks,… and new version 8.9.2-community and I am getting this message now:

RROR web[][o.s.s.p.PlatformImpl] Web server startup failed: Current version is too old. Please upgrade to Long Term Supportversion firstly.

However, I tried with different LTS versions and other regular versions and i am still getting same error…

DefinitelyNotTobi · August 24, 2021, 11:32am

I did not use a mssql database but a postgres db.

I have no idea what you do when creating fresh pvcs and a new database but the error message is indicating that the database contains sonarqube data from a old version of sonarqube.
Id advise that you manually check that there is no data in the environment you are bootstraping

Xiny · August 24, 2021, 11:45am

yes, i did it work finally re creating once again the database. I will try upgrading to new version to see if it works. thanks.

DefinitelyNotTobi · August 24, 2021, 6:25pm

best of luck

Xiny · August 25, 2021, 10:12am

ok the problem was here:

volumeMounts:
          - mountPath: "/opt/sonarqube/data/"
            name: sonar-data-new
          - mountPath: "/opt/sonarqube/extensions/"
            name: sonar-extensions-new

I had to change the extensions path for:

volumeMounts:
          - mountPath: "/opt/sonarqube/data/"
            name: sonar-data-new
          - mountPath: "/opt/sonarqube/extensions/plugins/"
            name: sonar-extensions-new

and now it works!

Not sure why i had to include the “plugins” on the path but looking on how some people create the containers for sonarqube i tried it and now it’s running fine.

system · September 1, 2021, 10:12am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.