Fail to execute es requestES indices exists request on indices 'metadatas'

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
  • sonarqube-10.3.0.82913, jdk-17
  • how is SonarQube deployed: zip, Docker, Helm
  • zip
  • what are you trying to achieve
  • trying to upgrade sonarquebe from ‘8.9.7’ to ‘10.3.0.82913’ and Start sonarqube server.
  • what have you tried so far to achieve this
  • stopped sonar 8.9.7.
  • downloaded sonarqube-10.3.0.82913 and updated systemd sonar service file with downloaded version.
  • Provided necessary folder permissions.
  • reloaded systemd sonar service file and enabled sonar
    Launching the server from the terminal and getting: Process exited with exit value [es]: 143

Tried removing data/es folder as well but didn’t work.

See below. Any help will be appreciated.
Thanks

Running SonarQube…

2024.01.23 18:51:09 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/apps/sonarqube-10.3.0.82913/temp
2024.01.23 18:51:09 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:40476]
2024.01.23 18:51:09 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/apps/sonarqube-10.3.0.82913/elasticsearch]: /opt/jdk-17.0.6.0.10/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=./bin/elasticsearch -Dcli.libs=lib/tools/server-cli -Des.path.home=/opt/apps/sonarqube-10.3.0.82913/elasticsearch -Des.path.conf=/opt/apps/sonarqube-10.3.0.82913/temp/conf/es -Des.distribution.type=tar -cp /opt/apps/sonarqube-10.3.0.82913/elasticsearch/lib/*:/opt/apps/sonarqube-10.3.0.82913/elasticsearch/lib/cli-launcher/* org.elasticsearch.launcher.CliToolLauncher
2024.01.23 18:51:09 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] Stopping SonarQube
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] Sonarqube has been requested to stop
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] Stopping [Compute Engine] process...
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] Stopping [Web Server] process...
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] Stopping [ElasticSearch] process...
2024.01.23 18:51:12 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 143
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2024.01.23 18:51:12 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped  

es.log

2024.01.23 18:51:12 INFO  es[][o.e.n.Node] JVM home [/opt/jdk-17.0.6.0.10], using bundled JDK [false]
2024.01.23 18:51:12 INFO  es[][o.e.n.Node] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=/opt/apps/sonarqube-10.3.0.82913/temp, -XX:ErrorFile=/opt/apps/sonarqube-10.3.0.82913/logs/es_hs_err_pid%p.log, -Xlog:disable, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/opt/apps/sonarqube-10.3.0.82913/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/opt/apps/sonarqube-10.3.0.82913/elasticsearch/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
2024.01.23 18:51:57 INFO  es[][o.e.n.Node] version[8.11.0], pid[18864], build[tar/d9ec3fa628c7b0ba3d25692e277ba26814820b20/2023-11-04T10:04:57.184859352Z], OS[Linux/3.10.0-957.12.2.el7.x86_64/amd64], JVM[Red Hat, Inc./OpenJDK 64-Bit Server VM/17.0.6/17.0.6+10-LTS]
2024.01.23 18:51:57 INFO  es[][o.e.n.Node] JVM home [/opt/jdk-17.0.6.0.10], using bundled JDK [false]
2024.01.23 18:51:57 INFO  es[][o.e.n.Node] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=/opt/apps/sonarqube-10.3.0.82913/temp, -XX:ErrorFile=/opt/apps/sonarqube-10.3.0.82913/logs/es_hs_err_pid%p.log, -Xlog:disable, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/opt/apps/sonarqube-10.3.0.82913/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/opt/apps/sonarqube-10.3.0.82913/elasticsearch/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [parent-join]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-core]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [old-lucene-versions]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [lang-painless]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [rest-root]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-redact]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [reindex]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [analysis-common]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-security]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [transport-netty4]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [aggregations]
2024.01.23 18:51:58 INFO  es[][o.e.p.PluginsService] loaded module [apm]
2024.01.23 18:52:03 INFO  es[][o.e.n.Node] version[8.11.0], pid[19028], build[tar/d9ec3fa628c7b0ba3d25692e277ba26814820b20/2023-11-04T10:04:57.184859352Z], OS[Linux/3.10.0-957.12.2.el7.x86_64/amd64], JVM[Red Hat, Inc./OpenJDK 64-Bit Server VM/17.0.6/17.0.6+10-LTS]

web.log

2024.01.23 18:51:49 INFO  web[][o.s.s.p.DetectPluginChange] Detect plugin changes
2024.01.23 18:51:50 INFO  web[][o.h.v.i.util.Version] HV000001: Hibernate Validator null
2024.01.23 18:51:50 WARN  web[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27-org.sonar.server.es.IndexCreator': Initialization of bean failed; nested exception is org.sonar.server.es.ElasticsearchException: Fail to execute es requestES indices exists request on indices 'metadatas'
2024.01.23 18:51:50 ERROR web[][o.s.s.p.Platform] Background initialization failed. Stopping SonarQube
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27-org.sonar.server.es.IndexCreator': Initialization of bean failed; nested exception is org.sonar.server.es.ElasticsearchException: Fail to execute es requestES indices exists request on indices 'metadatas'
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:200)
	at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
	at org.sonar.server.platform.platformlevel.PlatformLevel4.start(PlatformLevel4.java:670)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:214)
	at org.sonar.server.platform.PlatformImpl.startLevel34Containers(PlatformImpl.java:194)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.runIfNotAborted(PlatformImpl.java:362)
	at org.sonar.server.platform.PlatformImpl$1.doRun(PlatformImpl.java:113)
	at org.sonar.server.platform.PlatformImpl$AutoStarterRunnable.run(PlatformImpl.java:346)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.sonar.server.es.ElasticsearchException: Fail to execute es requestES indices exists request on indices 'metadatas'
	at org.sonar.server.es.EsClient.execute(EsClient.java:336)
	at org.sonar.server.es.EsClient.indexExists(EsClient.java:191)
	at org.sonar.server.es.IndexCreator.start(IndexCreator.java:78)
	at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	... 17 common frames omitted
Caused by: org.elasticsearch.ElasticsearchException: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
	at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2695)
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2154)
	at org.elasticsearch.client.IndicesClient.exists(IndicesClient.java:1279)
	at org.sonar.server.es.EsClient.lambda$indexExists$20(EsClient.java:191)
	at org.sonar.server.es.EsClient.execute(EsClient.java:334)
	... 23 common frames omitted
Caused by: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
	at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.getValue(BaseFuture.java:257)
	at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:244)
	at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:75)
	at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2692)
	... 28 common frames omitted
Caused by: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	... 1 common frames omitted
2024.01.23 18:51:50 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2024.01.23 18:51:50 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2024.01.23 18:51:50 INFO  web[][o.s.s.p.d.EmbeddedDatabase] Embedded database stopped
2024.01.23 18:51:50 INFO  web[][o.s.s.app.WebServer] Web Server stopped

Hi,

You can’t upgrade from 8.9.* straight to 10.3.

Your upgrade path is:

8.9.7 → 9.9.3 → 10.3 (last step optional)

You may find these resources helpful:

 
HTH,
Ann