Installation Issue - "Fail to unzip plugin [python]"..."Operation not permitted"

Must-share information (formatted with Markdown):

  • which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
    • SonarQube 10.1.0 Community
  • how is SonarQube deployed: zip, Docker, Helm
    • Kubernetes (Not using Helm (Unable to due to architecture))
  • what are you trying to achieve
    • Setup SonarQube to test on a small scale before going bigger.
  • what have you tried so far to achieve this
    • Read through the installation documentation multiple times (I know only Helm is officially supported, but have essentially used the helm templates and adapted them to something I can use on my own infrastructure, otherwise I can use this at all)

Logs

2023.07.02 21:55:17 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2023.07.02 21:55:17 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on [HTTP: 127.0.0.1:9001, TCP: 127.0.0.1:35691]
2023.07.02 21:55:17 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[ELASTICSEARCH] from [/opt/sonarqube/elasticsearch]: /opt/java/openjdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=./bin/elasticsearch -Dcli.libs=lib/tools/server-cli -Des.path.home=/opt/sonarqube/elasticsearch -Des.path.conf=/opt/sonarqube/temp/conf/es -Des.distribution.type=tar -cp /opt/sonarqube/elasticsearch/lib/*:/opt/sonarqube/elasticsearch/lib/cli-launcher/* org.elasticsearch.launcher.CliToolLauncher
2023.07.02 21:55:17 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2023.07.02 21:55:20 INFO  es[][o.e.n.Node] version[8.7.0], pid[98], build[tar/09520b59b6bc1057340b55750186466ea715e30e/2023-03-27T16:31:09.816451435Z], OS[Linux/5.15.0-76-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.7/17.0.7+7]
2023.07.02 21:55:20 INFO  es[][o.e.n.Node] JVM home [/opt/java/openjdk], using bundled JDK [false]
2023.07.02 21:55:20 INFO  es[][o.e.n.Node] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=/opt/sonarqube/temp, -XX:ErrorFile=/opt/sonarqube/logs/es_hs_err_pid%p.log, -Xlog:disable, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djna.tmpdir=/opt/sonarqube/temp, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=COMPAT, -Dcom.redhat.fips=false, -Des.enforce.bootstrap.checks=true, -Xmx512m, -Xms512m, -XX:MaxDirectMemorySize=256m, -XX:+HeapDumpOnOutOfMemoryError, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/opt/sonarqube/elasticsearch/lib, --add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [aggregations]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [analysis-common]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [apm]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [blob-cache]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [lang-painless]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [old-lucene-versions]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [parent-join]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [reindex]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [transport-netty4]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-aggregate-metric]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-core]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-profiling]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] loaded module [x-pack-security]
2023.07.02 21:55:21 INFO  es[][o.e.p.PluginsService] no plugins loaded
2023.07.02 21:55:24 INFO  es[][o.e.e.NodeEnvironment] using [1] data paths, mounts [[/opt/sonarqube/data (10.0.1.1:/mnt/ZPool1/Kubernetes/cluster0/personal-19/sonarqube/data)]], net usable_space [4.7tb], net total_space [4.7tb], types [nfs4]
2023.07.02 21:55:24 INFO  es[][o.e.e.NodeEnvironment] heap size [512mb], compressed ordinary object pointers [true]
2023.07.02 21:55:24 INFO  es[][o.e.n.Node] node name [sonarqube], node ID [lKa0DhO4Ss2jrx089YgiuA], cluster name [sonarqube], roles [ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold, ingest, data_frozen]
2023.07.02 21:55:24 INFO  es[][o.e.x.p.ProfilingPlugin] Profiling is enabled
2023.07.02 21:55:24 INFO  es[][o.e.x.s.Security] Security is disabled
2023.07.02 21:55:24 INFO  es[][o.e.t.n.NettyAllocator] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=4mb, heap_size=512mb}]
2023.07.02 21:55:24 INFO  es[][o.e.i.r.RecoverySettings] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
2023.07.02 21:55:24 INFO  es[][o.e.d.DiscoveryModule] using discovery type [single-node] and seed hosts providers [settings]
2023.07.02 21:55:25 INFO  es[][o.e.n.Node] initialized
2023.07.02 21:55:25 INFO  es[][o.e.n.Node] starting ...
2023.07.02 21:55:25 INFO  es[][o.e.t.TransportService] publish_address {127.0.0.1:35691}, bound_addresses {127.0.0.1:35691}
2023.07.02 21:55:25 INFO  es[][o.e.b.BootstrapChecks] explicitly enforcing bootstrap checks
2023.07.02 21:55:25 WARN  es[][o.e.c.c.ClusterBootstrapService] this node is locked into cluster UUID [HDZkyFIkRi2sp_BveAdgVg] but [cluster.initial_master_nodes] is set to [sonarqube]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts; for further information see https://www.elastic.co/guide/en/elasticsearch/reference/8.7/important-settings.html#initial_master_nodes
2023.07.02 21:55:25 INFO  es[][o.e.c.s.MasterService] elected-as-master ([1] nodes joined)[_FINISH_ELECTION_, {sonarqube}{lKa0DhO4Ss2jrx089YgiuA}{nLZiwfHqTwyq1R1mRHYZJA}{sonarqube}{127.0.0.1}{127.0.0.1:35691}{cdfhilmrstw}{8.7.0} completing election], term: 4, version: 14, delta: master node changed {previous [], current [{sonarqube}{lKa0DhO4Ss2jrx089YgiuA}{nLZiwfHqTwyq1R1mRHYZJA}{sonarqube}{127.0.0.1}{127.0.0.1:35691}{cdfhilmrstw}{8.7.0}]}
2023.07.02 21:55:26 INFO  es[][o.e.c.s.ClusterApplierService] master node changed {previous [], current [{sonarqube}{lKa0DhO4Ss2jrx089YgiuA}{nLZiwfHqTwyq1R1mRHYZJA}{sonarqube}{127.0.0.1}{127.0.0.1:35691}{cdfhilmrstw}{8.7.0}]}, term: 4, version: 14, reason: Publication{term=4, version=14}
2023.07.02 21:55:26 INFO  es[][o.e.r.s.FileSettingsService] starting file settings watcher ...
2023.07.02 21:55:26 INFO  es[][o.e.r.s.FileSettingsService] file settings service up and running [tid=30]
2023.07.02 21:55:26 INFO  es[][o.e.h.AbstractHttpServerTransport] publish_address {127.0.0.1:9001}, bound_addresses {127.0.0.1:9001}
2023.07.02 21:55:26 INFO  es[][o.e.c.c.NodeJoinExecutor] node-join: [{sonarqube}{lKa0DhO4Ss2jrx089YgiuA}{nLZiwfHqTwyq1R1mRHYZJA}{sonarqube}{127.0.0.1}{127.0.0.1:35691}{cdfhilmrstw}{8.7.0}] with reason [completing election]
2023.07.02 21:55:26 INFO  es[][o.e.n.Node] started {sonarqube}{lKa0DhO4Ss2jrx089YgiuA}{nLZiwfHqTwyq1R1mRHYZJA}{sonarqube}{127.0.0.1}{127.0.0.1:35691}{cdfhilmrstw}{8.7.0}{xpack.installed=true, rack_id=sonarqube}
2023.07.02 21:55:26 INFO  es[][o.e.l.LicenseService] license [bdd74327-7981-4b82-af56-5b0a01c50f5a] mode [basic] - valid
2023.07.02 21:55:26 INFO  es[][o.e.g.GatewayService] recovered [0] indices into cluster_state
2023.07.02 21:55:26 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
2023.07.02 21:55:26 INFO  app[][o.s.a.ProcessLauncherImpl] Launch process[WEB_SERVER] from [/opt/sonarqube]: /opt/java/openjdk/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/temp -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -Dcom.redhat.fips=false -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Dhttp.nonProxyHosts=localhost|127.*|[::1] -cp ./lib/sonar-application-10.1.0.73491.jar:/opt/sonarqube/lib/jdbc/postgresql/postgresql-42.6.0.jar org.sonar.server.app.WebServer /opt/sonarqube/temp/sq-process9317971875660968072properties
2023.07.02 21:55:26 INFO  es[][o.e.h.n.s.HealthNodeTaskExecutor] Node [{sonarqube}{lKa0DhO4Ss2jrx089YgiuA}] is selected as the current health node.
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.sonar.process.PluginSecurityManager (file:/opt/sonarqube/lib/sonar-application-10.1.0.73491.jar)
WARNING: Please consider reporting this to the maintainers of org.sonar.process.PluginSecurityManager
WARNING: System::setSecurityManager will be removed in a future release
2023.07.02 21:55:26 INFO  web[][o.s.p.ProcessEntryPoint] Starting Web Server
2023.07.02 21:55:27 INFO  web[][o.s.s.p.LogServerVersion] SonarQube Server / 10.1.0.73491 / 53c01c35c264c7e3d76cf5fb955de406f36b115e
2023.07.02 21:55:27 INFO  web[][o.s.d.DefaultDatabase] Create JDBC data source for jdbc:postgresql://postgresql-service:5432/sonarqube
2023.07.02 21:55:27 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Starting...
2023.07.02 21:55:27 INFO  web[][c.z.h.p.HikariPool] HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@73a00e09
2023.07.02 21:55:27 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Start completed.
2023.07.02 21:55:28 INFO  web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /opt/sonarqube
2023.07.02 21:55:28 INFO  web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy C# Code Quality and Security / 9.3.0.71466 / e47cf88a6286a446a098754e5775535a330f58d7
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Clean as You Code / 2.0.0.334 / 424c67de80b14c8c1b168104a6749b9d280ef4f9
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Configuration detection fot Code Quality and Security / 1.2.0.267 / 4f37ba9ffb37a96d5883e52ad392ed32c5c6eaab
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Flex Code Quality and Security / 2.9.0.3375 / 3fb9c557effa2d3f6e4017984611e8dae60d030a
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Go Code Quality and Security / 1.13.0.4374 / dc92e7c248e31ecae6e18d94781588cc6264e517
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy HTML Code Quality and Security / 3.8.0.3510 / c8f3d4a50984c6a0b0699616f98ce81a9d2893db
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy IaC Code Quality and Security / 1.17.0.3976 / 5089f396ccda1f2ea0a6f2961b10042ea8c9cd69
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy JaCoCo / 1.3.0.1538 / 74a7798c7cea687c72ed9df40c93eb7ea2a58c49
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Java Code Quality and Security / 7.20.0.31692 / b40c64f1075774f7232608df6636586009ab8ab3
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy JavaScript/TypeScript/CSS Code Quality and Security / 10.3.1.21905 / f95f3d172103f7e438556705c96fb336b33900b4
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Kotlin Code Quality and Security / 2.15.0.2579 / 6ea5707d5a5600fc808653075f9d9cb6c21a4788
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy PHP Code Quality and Security / 3.30.0.9766 / 783200fd5af4e0f8ccbf027a97ebb5f377d5014f
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Python Code Quality and Security / 4.3.0.11660 / 7914a5be1ad88eab1ebd2a3c2f7e606404601e30
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Ruby Code Quality and Security / 1.13.0.4374 / dc92e7c248e31ecae6e18d94781588cc6264e517
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Scala Code Quality and Security / 1.13.0.4374 / dc92e7c248e31ecae6e18d94781588cc6264e517
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy Text Code Quality and Security / 2.1.0.1163 / 849cef77adc8d1a50235d2ef7005e23f67e2838a
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy VB.NET Code Quality and Security / 9.3.0.71466 / e47cf88a6286a446a098754e5775535a330f58d7
2023.07.02 21:55:29 INFO  web[][o.s.s.p.ServerPluginManager] Deploy XML Code Quality and Security / 2.8.1.4006 / 80e5629b173cd7a47c04e97cbe1a263db84ff844
2023.07.02 21:55:29 WARN  web[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@61a88b8c-org.sonar.server.plugins.ServerPluginManager': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-4.3.0.11660.jar to /opt/sonarqube/data/web/deploy/plugins/python
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@61a88b8c-org.sonar.server.plugins.ServerPluginManager': Initialization of bean failed; nested exception is java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-4.3.0.11660.jar to /opt/sonarqube/data/web/deploy/plugins/python
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:187)
	at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:80)
	at org.sonar.server.platform.platformlevel.PlatformLevel2.start(PlatformLevel2.java:101)
	at org.sonar.server.platform.PlatformImpl.start(PlatformImpl.java:214)
	at org.sonar.server.platform.PlatformImpl.startLevel2Container(PlatformImpl.java:186)
	at org.sonar.server.platform.PlatformImpl.init(PlatformImpl.java:80)
	at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:45)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4494)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4946)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:871)
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:795)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1332)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1322)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:871)
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:249)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:428)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:917)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.sonar.server.app.EmbeddedTomcat.start(EmbeddedTomcat.java:71)
	at org.sonar.server.app.WebServer.start(WebServer.java:55)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:97)
	at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:81)
	at org.sonar.server.app.WebServer.main(WebServer.java:104)
Caused by: java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-4.3.0.11660.jar to /opt/sonarqube/data/web/deploy/plugins/python
	at org.sonar.server.plugins.ServerPluginJarExploder.explode(ServerPluginJarExploder.java:60)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toArray(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toArray(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toList(Unknown Source)
	at org.sonar.server.plugins.ServerPluginManager.extractPlugins(ServerPluginManager.java:86)
	at org.sonar.server.plugins.ServerPluginManager.start(ServerPluginManager.java:65)
	at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	... 44 common frames omitted
Caused by: java.nio.file.FileSystemException: /opt/sonarqube/data/web/deploy/plugins/python/sonar-python-plugin-4.3.0.11660.jar: Operation not permitted
	at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
	at java.base/sun.nio.fs.UnixCopyFile.copyFile(Unknown Source)
	at java.base/sun.nio.fs.UnixCopyFile.copy(Unknown Source)
	at java.base/sun.nio.fs.UnixFileSystemProvider.copy(Unknown Source)
	at java.base/java.nio.file.Files.copy(Unknown Source)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:850)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:756)
	at org.sonar.server.plugins.ServerPluginJarExploder.explode(ServerPluginJarExploder.java:56)
	... 59 common frames omitted
2023.07.02 21:55:29 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown initiated...
2023.07.02 21:55:29 INFO  web[][c.z.h.HikariDataSource] HikariPool-1 - Shutdown completed.
2023.07.02 21:55:29 INFO  web[][o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2023.07.02 21:55:29 INFO  web[][o.s.p.ProcessEntryPoint] Hard stopping process
2023.07.02 21:55:29 INFO  app[][o.s.a.SchedulerImpl] Process[Web Server] is stopped
2023.07.02 21:55:29 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 143
2023.07.02 21:55:29 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2023.07.02 21:55:29 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped

I’m unable to find much information online about this or even really know what exactly to search for at the moment. It seems like it’s a file system permissions issue, however, I cannot find issues, going so far as to setting all initial container files at /opt/sonarqube/* to 777 with owner and group being sonarqube.

Adjacent?

I use NFS as my storage backend through K8s Persistent Volume Claims, and suspect that it maybe be related to the issues, but cannot determine why.

I see on the Install the server page:


This warning for storage-related setups makes me suspicious it has something to do with my storage setup, but again, cannot figure out why, or even why bind mounts using Docker directly would have issues. Also similar is Deploy SonarQube on Kubernetes mentioning known issues on Azure Kubernetes Service with the Fileshare PVC.

Debugging Attempts

Going through the logs:

Looking at: nested exception is java.lang.IllegalStateException: Fail to unzip plugin [python] /opt/sonarqube/lib/extensions/sonar-python-plugin-4.3.0.11660.jar to /opt/sonarqube/data/web/deploy/plugins/python

I can go to /opt/sonarqube/lib/extensions/ and see:

sonarqube@sonarqube-deployment-cff8497f8-5mbrb:/opt/sonarqube$ ls -AGhl lib/extensions
total 144M
-r-xr-xr-x 1 root 9.0M Jun 20 13:44 sonar-cayc-plugin-2.0.0.334.jar
-r-xr-xr-x 1 root  28K Jun 20 13:44 sonar-config-plugin-1.2.0.267.jar
-r-xr-xr-x 1 root 4.9M Jun 20 13:44 sonar-csharp-plugin-9.3.0.71466.jar
-r-xr-xr-x 1 root 1.2M Jun 20 13:44 sonar-flex-plugin-2.9.0.3375.jar
-r-xr-xr-x 1 root 7.7M Jun 20 13:44 sonar-go-plugin-1.13.0.4374.jar
-r-xr-xr-x 1 root 528K Jun 20 13:44 sonar-html-plugin-3.8.0.3510.jar
-r-xr-xr-x 1 root 1.5M Jun 20 13:44 sonar-iac-plugin-1.17.0.3976.jar
-r-xr-xr-x 1 root  23K Jun 20 13:44 sonar-jacoco-plugin-1.3.0.1538.jar
-r-xr-xr-x 1 root  18M Jun 20 13:44 sonar-java-plugin-7.20.0.31692.jar
-r-xr-xr-x 1 root  20M Jun 20 13:44 sonar-javascript-plugin-10.3.1.21905.jar
-r-xr-xr-x 1 root  36M Jun 20 13:44 sonar-kotlin-plugin-2.15.0.2579.jar
-r-xr-xr-x 1 root 5.5M Jun 20 13:44 sonar-php-plugin-3.30.0.9766.jar
-r-xr-xr-x 1 root 8.0M Jun 20 13:44 sonar-python-plugin-4.3.0.11660.jar
-r-xr-xr-x 1 root  14M Jun 20 13:44 sonar-ruby-plugin-1.13.0.4374.jar
-r-xr-xr-x 1 root  13M Jun 20 13:44 sonar-scala-plugin-1.13.0.4374.jar
-r-xr-xr-x 1 root 112K Jun 20 13:44 sonar-text-plugin-2.1.0.1163.jar
-r-xr-xr-x 1 root 4.3M Jun 20 13:44 sonar-vbnet-plugin-9.3.0.71466.jar
-r-xr-xr-x 1 root 2.4M Jun 20 13:44 sonar-xml-plugin-2.8.1.4006.jar

All files are readable to the sonarqube user.

I can go to /opt/sonarqube/data/web/deploy/plugins/ and see:

sonarqube@sonarqube-deployment-cff8497f8-5mbrb:/opt/sonarqube$ stat /opt/sonarqube/data/web/deploy/plugins/python/
  File: /opt/sonarqube/data/web/deploy/plugins/python/
  Size: 2               Blocks: 1          IO Block: 1048576 directory
Device: 31h/49d Inode: 65074       Links: 2
Access: (0777/drwxrwxrwx)  Uid: ( 1000/sonarqube)   Gid: ( 1000/sonarqube)
Access: 2023-07-02 21:55:59.219114423 +0000
Modify: 2023-07-02 21:55:59.395114311 +0000
Change: 2023-07-02 22:35:24.553474386 +0000
 Birth: -

And so the directory is writeable to all users.

I believe the java application should be running under the sonarqube user, so I don’t see an issue there.

Declarations

Deployment (I’ll use StatefulSet eventually…)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sonarqube-deployment
  namespace: personal-19
  labels:
    app: sonarqube

spec:
  replicas: 1
  selector:
    matchLabels:
      app: sonarqube
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: sonarqube
        app-group: sonarqube
    spec:
      affinity:
        podAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              topologyKey: "kubernetes.io/hostname"
              labelSelector:
                matchExpressions:
                - key: app-group
                  operator: In
                  values:
                  - sonarqube
      initContainers:
      - name: set-sysctl-options
        image: docker.io/busybox:1.32
        command: [ "/bin/sh", "-c" ]
        args:
        - |
          if [[ "$(sysctl -n vm.max_map_count)" -lt 524288 ]]; then
            sysctl -w vm.max_map_count=524288
          fi
          if [[ "$(sysctl -n fs.file-max)" -lt 131072 ]]; then
            sysctl -w fs.file-max=131072
          fi
          if [[ "$(ulimit -n)" != "unlimited" ]]; then
            if [[ "$(ulimit -n)" -lt 131072 ]]; then
              echo "ulimit -n 131072"
              ulimit -n 131072
            fi
          fi
          if [[ "$(ulimit -u)" != "unlimited" ]]; then
            if [[ "$(ulimit -u)" -lt 8192 ]]; then
              echo "ulimit -u 8192"
              ulimit -u 8192
            fi
          fi
        securityContext:
          privileged: true
          # runAsUser: 1000
          # runAsGroup: 1000
          # readOnlyRootFilesystem: false
      - name: reset-filesystem-permissions
        image: docker.io/ubuntu:23.04
        command: [ "/bin/sh", "-c" ]
        args:
        - |
          chown -R 1000:1000 /opt/sonarqube
          chmod -R 777 /opt/sonarqube
        volumeMounts:
        - name: sonarqube-volume-data
          mountPath: /opt/sonarqube/data/
        - name: sonarqube-volume-extensions
          mountPath: /opt/sonarqube/extensions/
        - name: sonarqube-volume-logs
          mountPath: /opt/sonarqube/logs/
      containers:
      - name: sonarqube
        image: docker.io/sonarqube:10.1.0-community
        imagePullPolicy: Always
        # command: [ "sleep", "infinity" ]
        # securityContext:
          # privileged: true
          # runAsUser: 1000
          # runAsGroup: 1000
          # readOnlyRootFilesystem: false
        env:
        - name: SONAR_JDBC_URL
          value: 'jdbc:postgresql://postgresql-service:5432/sonarqube'
        - name: SONAR_JDBC_USERNAME
          valueFrom:
            secretKeyRef:
              name: personal-19-secrets
              key: Database-User
        - name: SONAR_JDBC_PASSWORD
          valueFrom:
            secretKeyRef:
              name: personal-19-secrets
              key: Database-Password
        ports:
        - protocol: TCP
          containerPort: 9000
        resources:
          requests:
            cpu: 500m
            memory: 2Gi
          limits:
            cpu: 4000m
            memory: 8Gi
        volumeMounts:
        - name: sonarqube-volume-data
          mountPath: /opt/sonarqube/data/
        - name: sonarqube-volume-extensions
          mountPath: /opt/sonarqube/extensions/
        - name: sonarqube-volume-logs
          mountPath: /opt/sonarqube/logs/
      terminationGracePeriodSeconds: 3600
      volumes:
      - name: sonarqube-volume-data
        persistentVolumeClaim:
          claimName: sonarqube-volumeclaim-data
      - name: sonarqube-volume-extensions
        persistentVolumeClaim:
          claimName: sonarqube-volumeclaim-extensions
      - name: sonarqube-volume-logs
        persistentVolumeClaim:
          claimName: sonarqube-volumeclaim-logs

Storage

apiVersion: v1
kind: PersistentVolume
metadata:
  name: personal-19-sonarqube-persistentvolume-data

spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  storageClassName: personal-19-sonarqube-storageclass-data
  volumeMode: Filesystem
  nfs:
    server: <local IP>
    path: /mnt/ZPool1/Kubernetes/cluster0/personal-19/sonarqube/data/
  mountOptions:
  - nfsvers=4.2
  - hard # `soft` may cause silent data corruption; `hard` requires requests to be retried indefinitely and is "not allowed to fail"
  - async # `async` can always be used, as application can command when data is flushed to disk (Basic write-caching)
  - proto=tcp
  - timeo=10 # read & write request timeout (in tenths of a second)
  - rsize=1048576 # the maximum number of bytes the client is allowed to request in a single read request
  - wsize=1048576 # the maximum number of bytes the client is allowed to request in a single write request
  - ac # enable file attribute caching
  - noatime # do not need to update file access times
  - nodiratime # do not need to update directory access times

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: personal-19-sonarqube-persistentvolume-extensions

spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  storageClassName: personal-19-sonarqube-storageclass-extensions
  volumeMode: Filesystem
  nfs:
    server: <local IP>
    path: /mnt/ZPool1/Kubernetes/cluster0/personal-19/sonarqube/extensions/
  mountOptions:
  - nfsvers=4.2
  - hard # `soft` may cause silent data corruption; `hard` requires requests to be retried indefinitely and is "not allowed to fail"
  - async # `async` can always be used, as application can command when data is flushed to disk (Basic write-caching)
  - proto=tcp
  - timeo=10 # read & write request timeout (in tenths of a second)
  - rsize=1048576 # the maximum number of bytes the client is allowed to request in a single read request
  - wsize=1048576 # the maximum number of bytes the client is allowed to request in a single write request
  - ac # enable file attribute caching
  - noatime # do not need to update file access times
  - nodiratime # do not need to update directory access times

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: personal-19-sonarqube-persistentvolume-logs

spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  storageClassName: personal-19-sonarqube-storageclass-logs
  volumeMode: Filesystem
  nfs:
    server: <local IP>
    path: /mnt/ZPool1/Kubernetes/cluster0/personal-19/sonarqube/logs/
  mountOptions:
  - nfsvers=4.2
  - hard # `soft` may cause silent data corruption; `hard` requires requests to be retried indefinitely and is "not allowed to fail"
  - async # `async` can always be used, as application can command when data is flushed to disk (Basic write-caching)
  - proto=tcp
  - timeo=10 # read & write request timeout (in tenths of a second)
  - rsize=1048576 # the maximum number of bytes the client is allowed to request in a single read request
  - wsize=1048576 # the maximum number of bytes the client is allowed to request in a single write request
  - ac # enable file attribute caching
  - noatime # do not need to update file access times
  - nodiratime # do not need to update directory access times

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sonarqube-volumeclaim-data
  namespace: personal-19

spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: personal-19-sonarqube-storageclass-data
  volumeMode: Filesystem

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sonarqube-volumeclaim-extensions
  namespace: personal-19

spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: personal-19-sonarqube-storageclass-extensions
  volumeMode: Filesystem

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sonarqube-volumeclaim-logs
  namespace: personal-19

spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  storageClassName: personal-19-sonarqube-storageclass-logs
  volumeMode: Filesystem

Ideas?

If anyone has any ideas on things I can look into, I would greatly appreciate any input! Thank you!

Dear @WilliamMiceli,

thanks for participating in the community!

First, we would like to ask a few questions about your preferred setup. You mention that you cannot install our helm chart out-of-the-box. Could you please clarify why?

Furthermore, are you required to use NFS? Do you encounter the same error when you don’t use NFS?

Hi Carmine,

For the part about not being able to use Helm, now looking into your Helm charts further, I probably could technically use them. Stating that I was definitively unable to use it was premature. Yours look very well done and does have places to insert things like scheduling preferences. Many of the applications I’ve looked at Helm for don’t always have very flexible charts, or include things like their own ingresses that I don’t have the ability to disable, when I already have Traefik in my cluster in-use for all public apps for HTTPS termination. And so I end up making my own manifest files for most apps anyway.

I would still have to make a few additional manifests for things like my Traefik setup, and of course storage, but then I’m making the manifest files anyway, so I’ve just gotten used to making my own.

I had actually found when going through your charts again that I had missed:

spec:
  template:
    spec:
      securityContext:
        fsGroup: 1000

Unfortunately it did not fix the storage issue though.

For needing to use NFS, I don’t have any hard requirements to use NFS, I can technically setup another system to use, but a while ago when I was doing just some very casual testing of different ways I could set things up, NFS provided the simplest configuration and best performance to my storage server. I’m sure I probably did plenty of things wrong when doing my testing in how I setup and used things, but I’ve come to like using NFS :stuck_out_tongue: And it’s mainly just a strong preference at this point.

I haven’t tried any other storage solutions just yet, but wouldn’t be against trying any out.

And although I am curious why using NFS doesn’t seem to work in this case (Maybe it’s something I need to fix and apply to my other apps), reading through everything again, this section in your documentation seems to have finally hit me:


I don’t know why my brain didn’t process it the first…second…third time, but I don’t actually need any of the volumes attached to SonarQube, just the database, which works just fine. And after removing the volume mounts to the SonarQube pod, it was of course able to start just fine. I had in my head that the volume mounts were essential, but if they aren’t required (as you’ve clearly stated), I have no issue with waiting longer for pods to start. I’d much rather do the more reliable way anyway.

I still might try another storage backend to see what the difference would be, and of course know if I was doing something wrong with NFS (probably very likely :stuck_out_tongue: ), but for now I’m happy I can finally try out SonarQube!