9.9.5-enterprise docker container fails to start

  • We try to update from SonarQube version: 9.9.4.87374 to 9.9.5-enterprise
  • SonarQube is running as a docker container

SonarQube 9.9.5-enterprise fails to start, with the docker logs containing:

...
2024.07.02 14:55:39 WARN  ce[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@6935070e-org.sonar.ce.container.CePluginRepository': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to unzip plugin [abap] /opt/sonarqube/lib/extensions/sonar-abap-plugin-3.11.0.4030.jar to /opt/sonarqube/temp/ce-exploded-plugins/abap
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@6935070e-org.sonar.ce.container.CePluginRepository': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to unzip plugin [abap] /opt/sonarqube/lib/extensions/sonar-abap-plugin-3.11.0.4030.jar to /opt/sonarqube/temp/ce-exploded-plugins/abap
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
        at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:187)
        at org.sonar.ce.container.ComputeEngineContainerImpl.startLevel2(ComputeEngineContainerImpl.java:211)
        at org.sonar.ce.container.ComputeEngineContainerImpl.start(ComputeEngineContainerImpl.java:184)
        at org.sonar.ce.app.CeServer$CeMainThread.attemptStartup(CeServer.java:164)
        at org.sonar.ce.app.CeServer$CeMainThread.run(CeServer.java:142)
Caused by: java.lang.IllegalStateException: Fail to unzip plugin [abap] /opt/sonarqube/lib/extensions/sonar-abap-plugin-3.11.0.4030.jar to /opt/sonarqube/temp/ce-exploded-plugins/abap
        at org.sonar.ce.container.CePluginJarExploder.explode(CePluginJarExploder.java:56)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
        at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
        at org.sonar.ce.container.CePluginRepository.extractPlugins(CePluginRepository.java:92)
        at org.sonar.ce.container.CePluginRepository.start(CePluginRepository.java:77)
        at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
        ... 14 common frames omitted
Caused by: java.io.IOException: Cannot create directory '/opt/sonarqube/temp/ce-exploded-plugins/abap'.
        at org.apache.commons.io.FileUtils.mkdirs(FileUtils.java:2200)
        at org.apache.commons.io.FileUtils.createParentDirectories(FileUtils.java:1113)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:843)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:756)
        at org.sonar.ce.container.CePluginJarExploder.explode(CePluginJarExploder.java:52)
        ... 27 common frames omitted
2024.07.02 14:55:39 INFO  ce[][o.s.p.ProcessEntryPoint] Hard stopping process
2024.07.02 14:55:39 INFO  app[][o.s.a.SchedulerImpl] Process[Compute Engine] is stopped
2024.07.02 14:55:39 INFO  web[][o.s.p.ProcessEntryPoint] Hard stopping process
2024.07.02 14:55:39 INFO  web[][o.s.s.e.CoreExtensionStopper] Stopping Governance
2024.07.02 14:55:39 INFO  web[][o.s.s.e.CoreExtensionStopper] Stopping Governance (done) | time=0ms
2024.07.02 14:55:39 INFO  web[][o.s.s.n.NotificationDaemon] Notification service stopped
2024.07.02 14:55:39 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2024.07.02 14:55:39 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2024.07.02 14:55:39 WARN  web[][o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [HikariPool-1 connection closer] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:\n java.base@17.0.11/java.util.concurrent.locks.LockSupport.unpark(Unknown Source)\n java.base@17.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.signalNext(Unknown Source)\n java.base@17.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.release(Unknown Source)\n java.base@17.0.11/java.util.concurrent.locks.ReentrantLock.unlock(Unknown Source)\n java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor.tryTerminate(Unknown Source)\n java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor.processWorkerExit(Unknown Source)\n java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n java.base@17.0.11/java.lang.Thread.run(Unknown Source)
2024.07.02 14:55:39 INFO  app[][o.s.a.SchedulerImpl] Process[Web Server] is stopped
2024.07.02 14:55:40 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 143
2024.07.02 14:55:40 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2024.07.02 14:55:40 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

We have found out that the issue was caused by the filesystem, docker daemon have tried to write the files, when the plugins were unpacked got filled up. Cleaning it up solved the problem.