Hello Folks
I have upgraded the sonarqube from 8.8.0.42792 to 9.9 LTS.
Its on centOS server. database is postgresql 12.6
Before upgrade, I have upgrade Java to v 17 as per the requirement.
openjdk 17.0.1 2021-10-19 LTS
OpenJDK Runtime Environment 21.9 (build 17.0.1+12-LTS)
OpenJDK 64-Bit Server VM 21.9 (build 17.0.1+12-LTS, mixed mode, sharing)
Upon starting sonarqube, it starts but then fails straightaway.
I have matched the sonar.properties file from the previous version. didn’t copy the config but matched the values.
es.log looks like this:
2024.01.17 17:12:26 INFO es[][o.e.n.Node] version[7.17.8], pid[674159], build[default/tar/120eabe1c8a0cb2ae87cffc109a5b65d213e9df1/2022-12-02T17:33:09.727072865Z], OS[Linux/4.18.0-348.7.1.el8_5.x86_64/amd64], JVM[Red Hat, Inc./OpenJDK 64-Bit Server VM/17.0.1/17.0.1+12-LTS]
2024.01.17 17:12:26 INFO es[][o.e.n.Node] JVM home [/usr/lib/jvm/java-17-openjdk-17.0.1.0.12-2.el8_5.x86_64]
2024.01.17 17:12:26 INFO es[][o.e.n.Node] JVM arguments [-XX:+UseG1GC, -Djava.io.tmpdir=/var/sonarqube/temp, -XX:ErrorFile=/opt/sonarqube/logs/es_hs_err_pid%p.log, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/var/sonarqube/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Des.enforce.bootstrap.checks=true, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/opt/sonarqube/elasticsearch, -Des.path.conf=/var/sonarqube/temp/conf/es, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=false]
2024.01.17 17:12:27 INFO es[][o.e.p.PluginsService] loaded module [analysis-common]
2024.01.17 17:12:27 INFO es[][o.e.p.PluginsService] loaded module [lang-painless]
2024.01.17 17:12:27 INFO es[][o.e.p.PluginsService] loaded module [parent-join]
2024.01.17 17:12:27 INFO es[][o.e.p.PluginsService] loaded module [reindex]
2024.01.17 17:12:27 INFO es[][o.e.p.PluginsService] loaded module [transport-netty4]
2024.01.17 17:12:27 INFO es[][o.e.p.PluginsService] no plugins loaded
2024.01.17 17:12:27 INFO es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/ (/dev/mapper/cl_bs--colo--postgresql-root)]], net usable_space [43.3gb], net total_space [49.9gb], types [xfs]
2024.01.17 17:12:27 INFO es[][o.e.e.NodeEnvironment] heap size [512mb], compressed ordinary object pointers [true]
2024.01.17 17:12:27 INFO es[][o.e.n.Node] node name [sonarqube], node ID [YGjXucMbSfukeisvmDUWFA], cluster name [sonarqube], roles [data_frozen, master, remote_cluster_client, data, data_content, data_hot, data_warm, data_cold, ingest]
2024.01.17 17:12:31 INFO es[][o.e.t.NettyAllocator] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=256kb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=512mb}]
2024.01.17 17:12:31 INFO es[][o.e.i.r.RecoverySettings] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
2024.01.17 17:12:31 INFO es[][o.e.d.DiscoveryModule] using discovery type [zen] and seed hosts providers [settings]
2024.01.17 17:12:31 INFO es[][o.e.g.DanglingIndicesState] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
2024.01.17 17:12:31 INFO es[][o.e.n.Node] initialized
2024.01.17 17:12:31 INFO es[][o.e.n.Node] starting ...
2024.01.17 17:12:31 INFO es[][o.e.t.TransportService] publish_address {127.0.0.1:41281}, bound_addresses {127.0.0.1:41281}
2024.01.17 17:12:32 INFO es[][o.e.b.BootstrapChecks] explicitly enforcing bootstrap checks
2024.01.17 17:12:32 INFO es[][o.e.c.c.Coordinator] cluster UUID [60OrpRjnTWWjRDqOQb3aDg]
2024.01.17 17:12:32 INFO es[][o.e.c.s.MasterService] elected-as-master ([1] nodes joined)[{sonarqube}{YGjXucMbSfukeisvmDUWFA}{vaWCr7TERF2LXAntJou7CQ}{127.0.0.1}{127.0.0.1:41281}{cdfhimrsw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 11, version: 263, delta: master node changed {previous [], current [{sonarqube}{YGjXucMbSfukeisvmDUWFA}{vaWCr7TERF2LXAntJou7CQ}{127.0.0.1}{127.0.0.1:41281}{cdfhimrsw}]}
2024.01.17 17:12:32 INFO es[][o.e.c.s.ClusterApplierService] master node changed {previous [], current [{sonarqube}{YGjXucMbSfukeisvmDUWFA}{vaWCr7TERF2LXAntJou7CQ}{127.0.0.1}{127.0.0.1:41281}{cdfhimrsw}]}, term: 11, version: 263, reason: Publication{term=11, version=263}
2024.01.17 17:12:32 INFO es[][o.e.h.AbstractHttpServerTransport] publish_address {127.0.0.1:9001}, bound_addresses {127.0.0.1:9001}
2024.01.17 17:12:32 INFO es[][o.e.n.Node] started
2024.01.17 17:12:32 INFO es[][o.e.g.GatewayService] recovered [7] indices into cluster_state
2024.01.17 17:12:34 INFO es[][o.e.c.r.a.AllocationService] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[metadatas][0], [components][1], [components][0]]]).
2024.01.17 17:12:37 INFO es[][o.e.n.Node] stopping ...
2024.01.17 17:12:37 INFO es[][o.e.n.Node] stopped
2024.01.17 17:12:37 INFO es[][o.e.n.Node] closing ...
2024.01.17 17:12:37 INFO es[][o.e.n.Node] closed
sonar.log looks like this:
2024.01.17 16:49:35 INFO app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2024.01.17 16:49:35 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped
2024.01.17 16:49:35 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2024.01.17 16:50:11 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /var/sonarqube/temp
2024.01.17 16:50:12 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:46817]
2024.01.17 16:50:12 ERROR app[][o.s.a.p.ManagedProcessHandler] Failed to launch process [ElasticSearch]
java.lang.IllegalStateException: Could not delete Elasticsearch temporary conf directory
at org.sonar.application.ProcessLauncherImpl.pruneElasticsearchConfDirectory(ProcessLauncherImpl.java:168)
at org.sonar.application.ProcessLauncherImpl.writeConfFiles(ProcessLauncherImpl.java:155)
at org.sonar.application.ProcessLauncherImpl.launch(ProcessLauncherImpl.java:92)
at org.sonar.application.SchedulerImpl.lambda$tryToStartProcess$2(SchedulerImpl.java:192)
at org.sonar.application.process.ManagedProcessHandler.start(ManagedProcessHandler.java:76)
at org.sonar.application.SchedulerImpl.tryToStartProcess(SchedulerImpl.java:190)
at org.sonar.application.SchedulerImpl.tryToStartEs(SchedulerImpl.java:142)
at org.sonar.application.SchedulerImpl.tryToStartAll(SchedulerImpl.java:134)
at org.sonar.application.SchedulerImpl.schedule(SchedulerImpl.java:113)
at org.sonar.application.App.start(App.java:59)
at org.sonar.application.App.main(App.java:81)
Caused by: java.nio.file.AccessDeniedException: /var/sonarqube/temp/conf/es
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:249)
at java.base/sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:110)
at java.base/java.nio.file.Files.deleteIfExists(Files.java:1191)
at org.sonar.application.ProcessLauncherImpl.pruneElasticsearchConfDirectory(ProcessLauncherImpl.java:166)
... 10 common frames omitted
2024.01.17 16:50:12 INFO app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2024.01.17 16:50:12 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped
2024.01.17 16:50:12 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2024.01.17 17:12:24 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /var/sonarqube/temp
2024.01.17 17:12:24 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:41281]
2024.01.17 17:12:24 INFO app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
2024.01.17 17:12:24 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2024.01.17 17:12:34 INFO app[][o.s.a.SchedulerImpl] Process[es] is up
2024.01.17 17:12:34 INFO app[][o.s.a.ProcessLauncherImpl] Launch process[WEB_SERVER] from [/opt/sonarqube]: /usr/lib/jvm/java-17-openjdk-17.0.1.0.12-2.el8_5.x86_64/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/var/sonarqube/temp -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -Dcom.redhat.fips=false -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttp.nonProxyHosts=localhost|127.*|[::1] -cp ./lib/sonar-application-9.9.3.79811.jar:/opt/sonarqube/lib/jdbc/postgresql/postgresql-42.5.1.jar org.sonar.server.app.WebServer /var/sonarqube/temp/sq-process17868514770088329943properties
2024.01.17 17:12:37 INFO app[][o.s.a.SchedulerImpl] Process[Web Server] is stopped
2024.01.17 17:12:37 WARN app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 143
2024.01.17 17:12:37 INFO app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2024.01.17 17:12:37 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped
web.log looks like this:
2024.01.17 16:21:12 INFO web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2024.01.17 16:21:12 INFO web[][o.s.p.ProcessEntryPoint] Hard stopping process
2024.01.17 17:12:34 INFO web[][o.s.p.ProcessEntryPoint] Starting Web Server
2024.01.17 17:12:36 INFO web[][o.s.s.p.LogServerVersion] SonarQube Server / 9.9.3.79811 / 4fefd8061afc12c43078b4cee31db2e4618b4fd6
2024.01.17 17:12:36 INFO web[][o.sonar.db.Database] Create JDBC data source for jdbc:postgresql://localhost/sonar
2024.01.17 17:12:36 INFO web[][c.z.h.HikariDataSource] HikariPool-1 - Starting...
2024.01.17 17:12:36 INFO web[][c.z.h.p.HikariPool] HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@5e3a39cd
2024.01.17 17:12:36 INFO web[][c.z.h.HikariDataSource] HikariPool-1 - Start completed.
2024.01.17 17:12:37 INFO web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /opt/sonarqube
2024.01.17 17:12:37 INFO web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled
2024.01.17 17:12:37 WARN web[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@5bc2b487-org.sonar.server.platform.DatabaseServerCompatibility': Initialization of bean failed; nested exception is Current version is too old. Please upgrade to Long Term Support version firstly.
2024.01.17 17:12:37 ERROR web[][o.s.s.p.PlatformImpl] Web server startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@5bc2b487-org.sonar.server.platform.DatabaseServerCompatibility': Initialization of bean failed; nested exception is Current version is too old. Please upgrade to Long Term Support version firstly.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:187)
at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
at org.sonar.server.platform.platformlevel.PlatformLevel2.start(PlatformLevel2.java:103)
at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:196)
at org.sonar.server.platform.PlatformImpl.startLevel2Container(PlatformImpl.java:169)
at org.sonar.server.platform.PlatformImpl.init(PlatformImpl.java:77)
at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:43)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4462)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4914)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:866)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:794)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:145)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:866)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:248)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:433)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:921)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
at org.apache.catalina.startup.Tomcat.start(Tomcat.java:489)
at org.sonar.server.app.EmbeddedTomcat.start(EmbeddedTomcat.java:72)
at org.sonar.server.app.WebServer.start(WebServer.java:55)
at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:97)
at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:81)
at org.sonar.server.app.WebServer.main(WebServer.java:104)
Caused by: org.sonar.api.utils.MessageException: Current version is too old. Please upgrade to Long Term Support version firstly.
2024.01.17 17:12:37 INFO web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2024.01.17 17:12:37 INFO web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2024.01.17 17:12:37 INFO web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2024.01.17 17:12:37 INFO web[][o.s.p.ProcessEntryPoint] Hard stopping process