SonarQube 8-community is not starting

SonarQube
1

Hi, Everyone.

Anyone has the same issue with me? I’m currently using Sonarqube 8-community, we were required by our Devops manager to upgrade it or remove the vulnerability related to log4j.

We mitigated the issue using removal of log4j in the Jndilookup class. Sonarqube is working as expected after it.

Just this month, sonarqube started to fail, it keeps on restarting with the message below.

Here is our docker-compose yaml file.
[root@apa-gittst-01 sonarqube]# cat docker-compose.yml
version: ‘3.3’

services:
  postgresql:
    image: postgres
    restart: always
    volumes:
      - $PWD/db:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: sonar
      POSTGRES_PASSWORD: sonar
    volumes:
      - ./postgresql:/var/lib/postgresql
      - ./postgresql_data:/var/lib/postgresql/data
  sonarqube:
    image: sonarqube:8-community
    restart: always
    ports:
      - 9000:9000
      - 9092:9092
    volumes:
      - ./sonarqube_conf:/opt/sonarqube/conf
      - ./sonarqube_data:/opt/sonarqube/data
      - ./sonarqube_extensions:/opt/sonarqube/extensions
      - ./sonarqube_bundled-plugins:/opt/sonarqube/lib/bundled-plugins
    environment:
      SONARQUBE_JDBC_USERNAME: sonar
      SONARQUBE_JDBC_PASSWORD: sonar
      SONARQUBE_JDBC_URL: jdbc:postgresql://postgresql/sonar
    depends_on:
      - postgresql
  adminer:
    image: adminer
    restart: always
    ports:
      - 7070:8080
    links:
      - postgresql:db
    depends_on:
      - postgresql

14:32:21.692 [main] WARN org.sonar.application.config.AppSettingsLoaderImpl - Configuration file not found: /opt/sonarqube/conf/sonar.properties
2022.06.20 14:32:21 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2022.06.20 14:32:21 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
2022.06.20 14:32:22 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
2022.06.20 14:32:22 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2022.06.20 14:32:22 INFO  app[][o.e.p.PluginsService] no modules loaded
2022.06.20 14:32:22 INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/config/properties/PropertiesConfigurationFactory
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.core.config.properties.PropertiesConfigurationFactory
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
        at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
        ... 1 more
2022.06.20 14:32:23 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 1
2022.06.20 14:32:23 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
2022.06.20 14:32:23 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

Any help will be appreciated. Thanks!

2

Hi @smendigoria2 ,

It looks like ElasticSearch is looking for a class in the log4j library that’s no longer there after your modifications. Any reason why you’re not just upgrading to the latest 8.9.8 LTS where this is fixed properly? The Jdnilookup class was only responsible for some of the recent log4j vulnerabilities; there was more after the initial one was discovered. Right now you’re in a state where you think you’re not vulnerable but most likely still are. Upgrading is the correct way to fix this.

Side note: You may want to re-think your Postgres password, especially after posting it here.

1 Like
3

Hi Chris,

I’ve tried upgrading and used the image 8.9.9-community from docker hub and seems like I’ve encountered an error.

10:30:50.363 [main] WARN org.sonar.application.config.AppSettingsLoaderImpl - Configuration file not found: /opt/sonarqube/conf/sonar.properties
2022.06.22 10:30:50 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2022.06.22 10:30:50 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:34072]
2022.06.22 10:30:50 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
2022.06.22 10:30:50 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
warning: no-jdk distributions that do not bundle a JDK are deprecated and will be removed in a future release
2022.06.22 10:30:53 INFO  es[][o.e.n.Node] version[7.16.2], pid[37], build[default/tar/2b937c44140b6559905130a8650c64dbd0879cfb/2021-12-18T19:42:46.604893745Z], OS[Linux/3.10.0-1160.66.1.el7.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/11.0.13/11.0.13+8]
2022.06.22 10:30:53 INFO  es[][o.e.n.Node] JVM home [/opt/java/openjdk]
2022.06.22 10:30:53 INFO  es[][o.e.n.Node] JVM arguments [-XX:+UseG1GC, -Djava.io.tmpdir=/opt/sonarqube/temp, -XX:ErrorFile=../logs/es_hs_err_pid%p.log, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/opt/sonarqube/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Des.enforce.bootstrap.checks=true, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/opt/sonarqube/elasticsearch, -Des.path.conf=/opt/sonarqube/temp/conf/es, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=false]
2022.06.22 10:30:54 INFO  es[][o.e.p.PluginsService] loaded module [analysis-common]
2022.06.22 10:30:54 INFO  es[][o.e.p.PluginsService] loaded module [lang-painless]
2022.06.22 10:30:54 INFO  es[][o.e.p.PluginsService] loaded module [parent-join]
2022.06.22 10:30:54 INFO  es[][o.e.p.PluginsService] loaded module [reindex]
2022.06.22 10:30:54 INFO  es[][o.e.p.PluginsService] loaded module [transport-netty4]
2022.06.22 10:30:54 INFO  es[][o.e.p.PluginsService] no plugins loaded
2022.06.22 10:30:54 INFO  es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/opt/sonarqube/data (/dev/sda2)]], net usable_space [13.9gb], net total_space [99.4gb], types [xfs]
2022.06.22 10:30:54 INFO  es[][o.e.e.NodeEnvironment] heap size [512mb], compressed ordinary object pointers [true]
**2022.06.22 10:30:54 ERROR es[][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main]**
**org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: cannot downgrade a node from version [7.17.1] to version [7.16.2]**
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.16.2.jar:7.16.2]
        at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.16.2.jar:7.16.2]
Caused by: java.lang.IllegalStateException: cannot downgrade a node from version [7.17.1] to version [7.16.2]
        at org.elasticsearch.env.NodeMetadata.upgradeToCurrentVersion(NodeMetadata.java:80) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:482) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:356) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.node.Node.<init>(Node.java:427) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.2.jar:7.16.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.2.jar:7.16.2]
        ... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: cannot downgrade a node from version [7.17.1] to version [7.16.2]
        at org.elasticsearch.env.NodeMetadata.upgradeToCurrentVersion(NodeMetadata.java:80)
        at org.elasticsearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:482)
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:356)
        at org.elasticsearch.node.Node.<init>(Node.java:427)
        at org.elasticsearch.node.Node.<init>(Node.java:309)
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234)
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434)
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
        at org.elasticsearch.cli.Command.main(Command.java:77)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
For complete error details, refer to the log at /opt/sonarqube/logs/sonarqube.log
2022.06.22 10:30:54 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 1
2022.06.22 10:30:54 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
2022.06.22 10:30:54 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
4

@smendigoria2 Any idea why you would have previously run ElasticSearch version 7.17.1? Did you maybe update the bundled version of ElasticSearch in your previous install of SonarQube in an attempt to fix the log4j issue?

I can’t find any documentation of which version of ElasticSearch is bundled with which version of SonarQube, you’ll have to wait for SQ support staff to take a look at this.

5

No I did not update the elastic search. I’m just new with sonarqube and trying to fix the problem to upgrade. All I did was to use new version of image in the docker-compose file.

6

Hi,

I searched on this part of the error message:

And found a thread where another user encountered a similar problem. The advice was:

Does that help?

 
Ann

7

Hi, Anne.

ES directory = elastic search?
And what cleaning do I need? Can you specify, thanks in advance!

8

Hi,

Yes, ‘ES’ == Elasticsearch.

To “clean” it, you just delete the contents.

 
HTH,
Ann