Can't deploy Sonarqube 8.5 to Azure Kubernetes

I’m able to deploy Sonarqube 8.3 and 8.4.1, but attempting to use the 8.5 docker image to deploy into Azure Kubernetes gives the following error on container startup:

2020.10.14 12:53:11 ERROR web[][o.s.s.p.PlatformImpl] Web server startup failed
java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-3.1.0.7619.jar to /opt/sonarqube/data/web/deploy/plugins/python
        at org.sonar.server.plugins.ServerPluginJarExploder.explode(ServerPluginJarExploder.java:60)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1675)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at org.sonar.server.plugins.ServerPluginManager.extractPlugins(ServerPluginManager.java:87)
        at org.sonar.server.plugins.ServerPluginManager.start(ServerPluginManager.java:66)
        at org.sonar.core.platform.StartableCloseableSafeLifecyleStrategy.start(StartableCloseableSafeLifecyleStrategy.java:40)
        at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)
        at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)
        at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)
        at org.picocontainer.behaviors.Stored.start(Stored.java:110)
        at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)
        at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)
        at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)
        at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
        at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:90)
        at org.sonar.server.platform.platformlevel.PlatformLevel2.start(PlatformLevel2.java:112)
        at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:213)
        at org.sonar.server.platform.PlatformImpl.startLevel2Container(PlatformImpl.java:179)
        at org.sonar.server.platform.PlatformImpl.init(PlatformImpl.java:87)
        at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:43)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4689)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5155)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1412)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1402)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: Failed setLastModified on /opt/sonarqube/lib/extensions/sonar-python-plugin-3.1.0.7619.jar
        at org.apache.commons.io.FileUtils.setLastModified(FileUtils.java:2561)
        at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:1400)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:885)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:835)
        at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:802)
        at org.sonar.server.plugins.ServerPluginJarExploder.explode(ServerPluginJarExploder.java:56)
        ... 33 common frames omitted
2020.10.14 12:53:12 INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2020.10.14 12:53:12 INFO  web[][o.s.p.ProcessEntryPoint] Hard stopping process
2020.10.14 12:53:12 INFO  app[][o.s.a.SchedulerImpl] Process[web] is stopped
2020.10.14 12:53:13 INFO  app[][o.s.a.SchedulerImpl] Process[es] is stopped
2020.10.14 12:53:13 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 143
2020.10.14 12:53:13 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
1 Like

Hi @Brian_Balderston ,

while this is not officially supported, i will try my best to help you on this one.

there was a recent change in the way plugins are handled in sonarqube. what kind of storage is underneath your PVs and what PVCs do you have in your deployment?

It’s using an Azure StorageV2 file share. I’ve attached the kubernetes deployment yaml (with the actual SONAR_JDBC_ values removed).sonarqube-k8.txt (1.6 KB)

I’m hitting the same.

I was able to install sonar in Azure AKS but with a lower sonarquebe image (mage: sonarqube:7.1). followed this post.

Hello…

Did you find any solution?

Hello everyone :wave:

i currently do not have a AKS cluster to validate this but can you look for a PodSecurityPolicy set that enforces a ReadOnlyRootFilesystem ?

we were reworking our plugins to have a fixed version of our analyzers be deployed with each release of sonarqube, so the stack trace indicates that there is a tried file operation to all analyzers in /opt/sonarqube/lib/extensions/ that is denied by said PSP.
The workaround would be to create a PVC for this directory and mount it in there (please be careful as all language analyzers are in there so copy the files to this new PVC) or to write a PSP that does not enforce a ReadOnlyRootFilesystem and start sonarqube in a pod that uses this new PSP.

Hope that helps

Hi…

The default value for readOnlyRootFilesystem is false. But I put it under securityContext anyway. I also tried fsGroup: 1000 changing the folder group to sonarqube. I also tried runAsUser: 1000 and runAsGroup: 1000.

Nothing above works.

But the problem happens only when unzipping to the folder /opt/sonarqube/data/web/deploy/plugins/python. I can see other files within /opt/sonarqube/data. And the volume mapped to /data was originally empty. Therefore, the folder has write permission.

Log: Failed to unzip the [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-3.1.0.7619.jar plugin to /opt/sonarqube/data/web/deploy/plugins/python

The same default Yaml configuration (without any of the changes above) works well with version 8.4.2.

If I change the version to 8.6.0 after a start the Sonarqube once with 8.4.2 version, it works fine. I guess that que data folder was alredy initialized by the 8.4.2 version.

Hi…

Creating the volumes using PersistentVolumeClaim do the trick. Works well with version 8.6.

The problem is actually related to the use of the storage account (file sharing).

Would be interesting see why previous version works fine with file share and 8.6 don’t. File share has some advantages, so much so that was the first choice of many people in this thread.

1 Like

I have the same issue, but with another plugin (CSS):
Fail to unzip plugin [cssfamily] /opt/sonarqube/lib/extensions/sonar-css-plugin-1.3.1.1642.jar to /opt/sonarqube/data/web/deploy/plugins/cssfamily
.

.
Caused by: java.io.IOException: Failed setLastModified on /opt/sonarqube/lib/extensions/sonar-css-plugin-1.3.1.1642.jar

Before I dive into creating beautiful new PV’s and PVC’s, I want to get something cleared up first.

My impression from the logfile is that the failure is occurring on the ‘/opt/sonarqube/lib/…’ path, not on the ‘/opt/sonarqube/data/…’ path. The former is part of the docker image right???

When I examine my file-store in Azure, I see that the ‘sonar-css-plugin-1.3.1.1642.jar’ is present in the expected path there…
My hunch is that the unzip works fine, but the ‘setLastModified’ on the source jar (present in the docker image) fails…

I think that the ‘solution’ of creating a persistent opt/sonarqube/lib/… storage and then manually copying data from the image into that share is a bit convoluted…

Thank you for listening!

Where might I find the analyzers as used by/included for SonarQube 8.6 developer?

@Tobias_Trabelsi Do you think this issue would be fixed in the next version of SonarQube?

@Tobias_Trabelsi Is the issue with azure file storage resolved by sonar because I am also facing this issue when I try to intall soanrqube developer edition in my linux machine with /data being mounted with azure file share.

Hello All,

This is my first post in this forum and first time using SONAR :smile:

I too was facing similar issue when installing SonarQube 9.3 on AKS. After spending a day worth of time, I was able to crack it. Details as follows:

PV/PVC created on :
Azure Fileshare (backed by Storage account - Premium/Hot)
Protocol - SMB, FileREST

Error:
2022.02.10 02:59:49 ERROR web[o.s.s.p.PlatformImpl] Web server startup failed
java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-3.9.0.9230.jar to /opt/sonarqube/data/web/deploy/plugins/python


Caused by: java.nio.file.FileSystemException: /opt/sonarqube/data/web/deploy/plugins/python/sonar-python-plugin-3.9.0.9230.jar: Operation not permitted

Probable Root Cause (I may be wrong since its my firstime with Sonarqube ) :
Sonar tries to write to fileshare (/opt/sonarqube/data) with its own non-root ownership (mostly uid-1000 / gid-1000) but the fileshare when mounted via PV/PVC is mounted using root ownership (uid - 0 / gid - 0). Hence, we cannot change ownership of a fileshare after its mounted (i.e chown -R 1000:1000 /opt/sonarqube/extensions" will not work)

Fix:
Change the PV mountOptions ownership to non-root user (uid-1000 / gid-1000) in pv.yaml file as below. This worked like charm for me and no more errors even while installing sonar plugins to /opt/sonarqube/data dir.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: sonarqube-data

spec:
  storageClassName:  azurefile-csi
  capacity:
    storage: "100Gi"
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy:
  mountOptions:
    - dir_mode=0777
    - file_mode=0777
    - uid=1000
    - gid=1000
    - mfsymlinks
    - cache=strict      - nosharesock
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: sonarqube-data
    namespace: app

  csi:
    driver: file.csi.azure.com
    readOnly: false
    volumeHandle: sonarstorage-sonarqube-data
    volumeAttributes:
      resourceGroup: sonar-rg  
      storageaccount: sonarstorage
      shareName: sonarqube-data  # only file share name, don't use full path
      server: "sonarstorage.privatelink.file.core.windows.net"
    nodeStageSecretRef:
      name: storage-secret
      namespace: default

Hope this saves some time for you all.

@Tobias_Trabelsi - Would like to know your view on this.

1 Like

I have the same problem using Azure Container Apps. The problem is I can’t use mountOptions like Kubernetes (there is no such configuration in Container Apps)

Hi @sfragata , can you share which image and tag you used to deploy SonarQube? Also, what logs or errors did you encounter that led you to this Sonar Community thread? Was it the same as the opening post of this thread?

Hi Joe
I’m using sonarqube:10.3.0-community
and the stacktrace is

2023.12.19 14:08:18 ERROR web[][o.s.s.p.w.PlatformServletContextListener] Web server startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@31bcf236-org.sonar.server.plugins.ServerPluginManager': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-4.10.0.13725.jar to /opt/sonarqube/data/web/deploy/plugins/python
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:200)
	at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
	at org.sonar.server.platform.platformlevel.PlatformLevel2.start(PlatformLevel2.java:101)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:214)
	at org.sonar.server.platform.PlatformImpl.startLevel2Container(PlatformImpl.java:186)
	at org.sonar.server.platform.PlatformImpl.init(PlatformImpl.java:80)
	at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:45)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4462)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4914)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:866)
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:794)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:866)
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:248)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:433)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:921)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171)
	at org.apache.catalina.startup.Tomcat.start(Tomcat.java:489)
	at org.sonar.server.app.EmbeddedTomcat.start(EmbeddedTomcat.java:72)
	at org.sonar.server.app.WebServer.start(WebServer.java:55)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:97)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:81)
	at org.sonar.server.app.WebServer.main(WebServer.java:104)
Caused by: java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-4.10.0.13725.jar to /opt/sonarqube/data/web/deploy/plugins/python
	at org.sonar.server.plugins.ServerPluginJarExploder.explode(ServerPluginJarExploder.java:60)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toArray(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toArray(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toList(Unknown Source)
	at org.sonar.server.plugins.ServerPluginManager.extractPlugins(ServerPluginManager.java:86)
	at org.sonar.server.plugins.ServerPluginManager.start(ServerPluginManager.java:65)
	at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	... 44 common frames omitted
Caused by: java.io.IOException: Cannot set the file time.
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:815)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:838)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:746)
	at org.sonar.server.plugins.ServerPluginJarExploder.explode(ServerPluginJarExploder.java:56)
	... 59 common frames omitted