Upgrade version via Docker Compose

Hello everyone and have a good day!

Is there any correct way to update sonarqube as part of the docker compose stack? I didn’t find such information in the official documentation

Hi,

Are you asking about how to upgrade SonarQube? The docs should help.

 
Ann

the documentation doesn’t say anything about compose.yml

II have an 8.1-community-beta image
I tried to update the image from versions from 8.2 to the latest.
I did it in the following way:

  1. change the tag in the sonarqube image in docker-compose.yml
  2. running

docker stack up -c docker-compose.yml sonarqube

nginx and postgres services go up, but sonarqube mostly gives an error:

bin/run.sh: line 6: /opt/sonarqube/bin/sonar.sh: No such file or directory

Hello @cryptoparty thanks a lot for taking the time to participate in the community.

Could you please provide us the docker-compose files ( previsous one, and updated one ) ?

In the meantime i am not sure if you are trying to update :

  • 8.1-community-beta => 8.2
  • 8.1-community-beta => sonarqube:latest ( which is 9.8 )
  • 8.2 => sonarqube:latest

Could you please clarify that part ?

Anyway your process is good, i would suggest to break each step like that:

  • docker-compose down sonarqube
  • #change the image version
  • docker-compose up sonarqube

But in the end this is what your command did so it should be fine !

I am waiting for you compose file in order to investigate a bit more, as the error message imply it could be related to some volumes.

Waiting for your input,

Jérémy.

Hi all,

i cannot add/help concerning the correct order of the version-upgrade-dance but…

i’d like to (advertise? :thinking: :person_shrugging: ) something concerning the volume remark jeremy mentioned.

I found relief in upgrading when i settled upon cloning the currently used volumes into “target-version-volumes” to be used in the new “target-version” compose-setup. This way i always can go back to the “untouched source-version-volumes” when something goes wronk :sweat_smile:

You can find helpful scripts for that here: docker-convenience-scripts/docker_clone_volume.sh at master · gdiepen/docker-convenience-scripts · GitHub

I explained my process here: Documentation - How to include 3rd Party Plugins in a Docker-Setup - #12 by daniel

cheers
Daniel

1 Like
version: "3.7"

services:
  sonarqube:
    image: sonarqube:8.1-community-beta
    hostname: sonarqube
    env_file:
      - sonarqube.env
    volumes:
      - sonarall:/opt/sonarqube
 #     - sonarqube_conf:/opt/sonarqube/conf
 #     - sonarqube_data:/opt/sonarqube/data
 #     - sonarqube_extensions:/opt/sonarqube/extensions
 #     - sonarqube_bundled-plugins:/opt/sonarqube/lib/bundled-plugins
    configs:
      - source: conf_plugin_swift
        target: /usr/bin/run-sonar-swift.sh
    networks:
      - net_sonar1
    deploy:
      endpoint_mode: dnsrr
    ports:
     - target: 9000
       published: 9000
       protocol: tcp
       mode: host

  db:
    image: postgres:12.1
    env_file:
      - postgres.env
    volumes:
      - postgresql:/var/lib/postgresql
      - postgresql_data:/var/lib/postgresql/data
    networks:
      - net_sonar1

  nginx:
    image: nginx:1.17.6-alpine
    depends_on:
      - sonarqube
    networks:
      - net_sonar1
    ports:
      - 80:80
      - 443:443
    configs:
      - source: conf_nginx
        target: /etc/nginx/conf.d/default.conf
    volumes:
      - nginx_log:/var/log/nginx
    secrets:
      - source: nginx_crt
        target: /etc/ssl/certs/sonarqube.crt #crt
      - source: nginx_key
        target: /etc/ssl/private/sonarqube.key #key

secrets:
  nginx_crt:
    external: true
  nginx_key:
    external: true

configs:
  conf_nginx:
    external: true
  conf_plugin_swift:
    external: true

networks:
  net_sonar1:
    driver: overlay
    driver_opts:
      encrypted: "true"

volumes:
#  sonarqube_conf:
#  sonarqube_data:
#  sonarqube_extensions:
#  sonarqube_bundled-plugins:
  postgresql:
  postgresql_data:
  nginx_log:
  sonarall:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /opt/volumes/newsonar

Hello @cryptoparty, by having the volume sonarall mounted to /opt/sonarqube It is erasing everything we have inside our images, which implies the entrypoint from you error message, but also the java files and the sonarqube application.

My statement above is not true with the 8.1 as the directory structure was not the same, but starting with 8.2 and every 9.x images this is the case.

Because sub-dir on named volume is not yet possible from what i know, you will have to declare individual one.

Hope that answer the question.

Jérémy.

Thanks for the answer, but it is not completely clear what you need to do to eliminate the error

Hello @cryptoparty i will try to clarify a little bit the situation.

In the sample, we have four different sub volumes, because the root /opt/sonarqube is already populated with the SonarQube application

In your case the easiest way to fix that, would be to split your sonarall volume into four different ones, while keeping your data.

For example the volume definition could look like this :

volumes:
  sonarqube_conf:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /opt/volumes/newsonar/conf
  sonarqube_data:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /opt/volumes/newsonar/data
  sonarqube_extensions:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /opt/volumes/newsonar/extensions
  sonarqube_bundled-plugins:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /opt/volumes/newsonar/lib/bundled-plugins

and then you would have to uncomment the sample volumes mount and upgrade the image to 8.2+

Side note : Please be careful with /opt/volumes/newsonar/lib/bundled-plugins on your disk, if that path does not exist, the docker-compose will fail.

Thank you, broke up the disks on your advice.
I added the latest tag and got the service log as a result:

sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | Dropping Privileges
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:55 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:56 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:43932]
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:56 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | could not find java in ES_JAVA_HOME at /usr/lib/jvm/java-17-openjdk/bin/java
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:56 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 1
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:56 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:56 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
sonarqube_sonarqube.1.r1np8owwzprc@s001ss-codereviewib    | 2023.01.09 06:16:56 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | Dropping Privileges
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:33190]
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | Dropping Privileges
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | could not find java in ES_JAVA_HOME at /usr/lib/jvm/java-17-openjdk/bin/java
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:38171]
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 1
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
sonarqube_sonarqube.1.sc99gjsgde1d@s001ss-codereviewib    | 2023.01.09 06:17:02 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | could not find java in ES_JAVA_HOME at /usr/lib/jvm/java-17-openjdk/bin/java
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | Dropping Privileges
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 1
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
sonarqube_sonarqube.1.34mucpozp25f@s001ss-codereviewib    | 2023.01.09 06:16:49 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:40805]
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | could not find java in ES_JAVA_HOME at /usr/lib/jvm/java-17-openjdk/bin/java
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 1
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
sonarqube_sonarqube.1.2fn394je3xes@s001ss-codereviewib    | 2023.01.09 06:17:09 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

When i get back 8.1-community-beta got the same errors.

sonarqube.env:

sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:postgresql://db/sonar

/etc/java/java.conf:

# System-wide Java configuration file  -*- sh -*-

# Location of jar files on the system
JAVA_LIBDIR=/usr/share/java

# Location of arch-specific jar files on the system
JNI_LIBDIR=/usr/lib/java

# Root of all JVM installations
JVM_ROOT=/usr/lib/jvm

# You can define a system-wide JVM root here if you're not using the
# default one.
#
# If you have a base JRE package installed
# (e.g. java-1.6.0-openjdk):
#JAVA_HOME=$JVM_ROOT/jre
#
# If you have a devel JDK package installed
# (e.g. java-1.6.0-openjdk-devel):
#JAVA_HOME=$JVM_ROOT/java

# Options to pass to the java interpreter
#JAVACMD_OPTS=

# You can disable ABRT Java Connector by setting JAVA_ABRT to "off".
# See: https://github.com/jfilak/abrt-java-connector/
#JAVA_ABRT=off

Hello @cryptoparty, be careful with changing JVM path, our SonarQube images already contains JVM installed to a specific path. i believe this is the issue here.

For 8.2-community the jvm is located in /usr/local/openjdk-11/bin/
For sonarqube:latest it is located in /usr/lib/jvm/java-17-openjdk

I dont know much about java but the JVM_ROOT could be an issue ?

When upgrading to 8.2-community - service went up!

docker ps
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                    NAMES
133b2d8827aa        **sonarqube:8.2-community**   "./bin/run.sh"           7 seconds ago       Up 1 second         0.0.0.0:9000->9000/tcp   sonarqube_sonarqube.1.u32jda14vlcw9ofroii5w9f9z
4cd1c0b0b2f0        nginx:1.17.6-alpine       "nginx -g 'daemon of…"   3 hours ago         Up 3 hours          80/tcp                   sonarqube_nginx.1.df60u5fdtssc01rnep5sudo97
a8d31cc26a09        postgres:12.1             "docker-entrypoint.s…"   3 hours ago         Up 3 hours          5432/tcp                 sonarqube_db.1.rqhjphy8yjtwboq9n6ooqhyn7

But it keeps restarting:

sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    | Caused by: java.nio.file.AccessDeniedException: /opt/sonarqube/data/es6
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6j057iua6skd@s001ss-codereviewib    | 2023.01.09 08:46:58 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 1
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6j057iua6skd@s001ss-codereviewib    | 2023.01.09 08:46:58 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at sun.nio.fs.UnixFileSystemProvider.createDirectory(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6j057iua6skd@s001ss-codereviewib    | 2023.01.09 08:46:58 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at java.nio.file.Files.createDirectory(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at java.nio.file.Files.createAndCheckIsDirectory(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at java.nio.file.Files.createDirectories(Unknown Source) ~[?:?]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:413) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:68) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:299) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:254) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Security.configure(Security.java:123) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:207) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.4.jar:6.8.4]
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    |     ... 6 more
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    | 2023.01.09 08:46:47 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 1
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    | 2023.01.09 08:46:47 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
sonarqube_sonarqube.1.6zyw0dlpngzg@s001ss-codereviewib    | 2023.01.09 08:46:47 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

The error seems to be different

image

From the log it looks like a file permission issue, by default SonarQube image use an user with uid 1000.

Could you double check on your disk that /opt/volumes/newsonar/data has the correct set of permissions ?

If this is a testing instance, quick check by doing a chmod -R 777 /opt/volumes/newsonar/data

If its an important instance, make sure uid 1000 can read and write to this folder.

Ps: this could happen for the other folder, you should double check on them also.

It was 0755 everywhere, added 0777 - it didn’t work.

I also want to replace that in the output I have 1002:

# id -u sonar
1002

Do I need to change Uid?

UPD: Changing Uid for user sonar didnt work same

hmm that’s strange, are the log showing the same error message now that you have 0777 as permissions ? or is it another error ?

same thing. Could it be that the catalogs already have data?

Could be yes, i would suggest in such cases to follow the documentation laying around here

Which include Drop the Elasticsearch indexes by deleting the contents of <SONARQUBE_HOME>/data/es6 directory.

Please let me know if this solves the issue.

Do I understand correctly that I need to delete /data/es6 and restart server, and backup in case of recovery?

Yes you can delete /data/es6 freely.

A database backup is always interesting before doing an upgrade.

The same errors:

Caused by: java.lang.IllegalStateException: Unable to access 'path.data' (/opt/sonarqube/data/es6)
Caused by: java.nio.file.AccessDeniedException: /opt/sonarqube/data/es6

Next failed to pick up and version 8.1-beta. After restoring the base, the error persists. Apparently, it is exactly in the access rights

additionally noticed that on a running instance, the permissions are 755 and the user polkitd and group input: