Error when trying to upgrade from the 9.9.0-developer to the 9.9.0-enterprise edition

TADIE_Denis_Laurel · November 7, 2023, 11:00am

Hello Everyone,

I want to upgrade from the 9.9.0-developer to the 9.9.0-enterprise edition. We are using the chart helm template to deploy our sonar instance on AKS but we are getting some errors. But when we rollback to the developer edition, there is no more errors.

CONFIGURATION
resources quota of the namespace:

requests.cpu: "4"
requests.memory: 4Gi
limits.cpu: "5"
limits.memory: 7Gi

: Java 17, Postgresql version 11

Thanks if you have any suggestions regarding these request.

Here is an extract of the related errors:

23.11.07 02:09:22 INFO  ce[][c.z.h.p.HikariPool] HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@77287bb3
2023.11.07 02:09:22 INFO  ce[][c.z.h.HikariDataSource] HikariPool-1 - Start completed.
2023.11.07 02:09:22 WARN  app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [ElasticSearch]: 137
2023.11.07 02:09:22 INFO  app[][o.s.a.SchedulerImpl] Process[ElasticSearch] is stopped
2023.11.07 02:09:22 INFO  ce[][o.s.p.ProcessEntryPoint] Hard stopping process
2023.11.07 02:09:23 INFO  ce[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /opt/sonarqube
2023.11.07 02:09:23 INFO  ce[][o.s.c.e.CoreExtensionsLoader] Loaded core extensions: enterprise-edition, ce-workers, developer-scanner, developer-server, governance, license, securityreport, monitoring, scim
2023.11.07 02:09:23 INFO  ce[][o.s.c.c.CePluginRepository] Load plugins
2023.11.07 02:09:23 WARN  ce[][o.s.p.ProcessEntryPoint$HardStopperThread] Can not stop in 1000ms
2023.11.07 02:09:25 WARN  ce[][o.s.c.a.AnnotationConfigApplicationContext] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@7401ec2b-org.sonar.ce.httpd.CeHttpServer': Initialization of bean failed; nested exception is java.lang.IllegalArgumentException: Unable to create shared memory : 
2023.11.07 02:09:25 INFO  ce[][o.s.s.e.CoreExtensionStopper] Stopping Governance
2023.11.07 02:09:25 INFO  ce[][o.s.s.e.CoreExtensionStopper] Stopping Governance (done) | time=1ms
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@7401ec2b-org.sonar.ce.httpd.CeHttpServer': Initialization of bean failed; nested exception is java.lang.IllegalArgumentException: Unable to create shared memory : 
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:628)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:187)
	at org.sonar.ce.container.ComputeEngineContainerImpl.startLevel4(ComputeEngineContainerImpl.java:227)
	at org.sonar.ce.container.ComputeEngineContainerImpl.start(ComputeEngineContainerImpl.java:192)
	at org.sonar.ce.app.CeServer$CeMainThread.attemptStartup(CeServer.java:164)
	at org.sonar.ce.app.CeServer$CeMainThread.run(CeServer.java:142)
Caused by: java.lang.IllegalArgumentException: Unable to create shared memory : 
	at org.sonar.process.sharedmemoryfile.AllProcessesCommands.<init>(AllProcessesCommands.java:103)
	at org.sonar.process.sharedmemoryfile.DefaultProcessCommands.<init>(DefaultProcessCommands.java:34)
	at org.sonar.process.sharedmemoryfile.DefaultProcessCommands.secondary(DefaultProcessCommands.java:52)
	at org.sonar.ce.httpd.CeHttpServer.registerServerUrl(CeHttpServer.java:79)
	at org.sonar.ce.httpd.CeHttpServer.start(CeHttpServer.java:62)
	at org.sonar.core.platform.StartableBeanPostProcessor.postProcessBeforeInitialization(StartableBeanPostProcessor.java:33)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	... 14 common frames omitted
Caused by: java.nio.channels.ClosedByInterruptException: null
	at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(Unknown Source)
	at java.base/sun.nio.ch.FileChannelImpl.endBlocking(Unknown Source)
	at java.base/sun.nio.ch.FileChannelImpl.mapInternal(Unknown Source)
	at java.base/sun.nio.ch.FileChannelImpl.map(Unknown Source)
	at org.sonar.process.sharedmemoryfile.AllProcessesCommands.<init>(AllProcessesCommands.java:101)
	... 22 common frames omitted
2023.11.07 02:09:25 ERROR web[][o.s.s.m.ElasticSearchMetricTask] Failed to query ES status
org.sonar.server.es.ElasticsearchException: Fail to execute es requestES cluster health request
	at org.sonar.server.es.EsClient.execute(EsClient.java:313)
	at org.sonar.server.es.EsClient.clusterHealth(EsClient.java:206)
	at org.sonar.server.monitoring.ElasticSearchMetricTask.updateElasticSearchHealthStatus(ElasticSearchMetricTask.java:56)
	at org.sonar.server.monitoring.ElasticSearchMetricTask.run(ElasticSearchMetricTask.java:50)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Connection refused
	at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:932)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:300)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
	at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2699)
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2137)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:2105)
	at org.elasticsearch.client.ClusterClient.health(ClusterClient.java:151)
	at org.sonar.server.es.EsClient.lambda$clusterHealth$27(EsClient.java:206)
	at org.sonar.server.es.EsClient.execute(EsClient.java:311)
	... 9 common frames omitted
Caused by: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	... 1 common frames omitted
2023.11.07 02:09:25 ERROR web[][o.s.s.m.ElasticSearchMetricTask] Failed to query ES status
org.sonar.server.es.ElasticsearchException: Fail to execute es request
	at org.sonar.server.es.EsClient.execute(EsClient.java:313)
	at org.sonar.server.es.EsClient.execute(EsClient.java:305)
	at org.sonar.server.es.EsClient.nodesStats(EsClient.java:216)
	at org.sonar.server.monitoring.ElasticSearchMetricTask.updateFileSystemMetrics(ElasticSearchMetricTask.java:77)
	at org.sonar.server.monitoring.ElasticSearchMetricTask.run(ElasticSearchMetricTask.java:51)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

TADIE_Denis_Laurel · November 8, 2023, 10:21pm

We managed to resolve this issue.

Explanation: We are using the sonarqube helm chart to deploy our application but we were using an old version which was not compatible with the LTS version 9.9-LTS. We saw that there were some misconfiguration in the old helm chart. But the misconfiguration related to our error where on the elasticsearch server which where running with this configuration:

elasticsearch:
  # DEPRECATED: Use initSysctl.enabled instead
  configureNode: false
  bootstrapChecks: false

And this is leading to check on the elasticsearch server being ignore. By changing this configuration to

elasticsearch:
  # DEPRECATED: Use initSysctl.enabled instead
  configureNode: false
  bootstrapChecks: true

We saw that the real error which was leading to the error in the description was:

# [Elasticsearch: Max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]](https://stackoverflow.com/questions/51445846/elasticsearch-max-virtual-memory-areas-vm-max-map-count-65530-is-too-low-inc)

Having this error instead of the previous one was pretty quite easy to solve since we just had to change the configuration configureNode to true to upgrade vm_max_coun of server.

system · November 15, 2023, 10:21pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.