Hello,
sonarqube start throwing error while running background tasks.
nothing much in logs except the same error as below:
org.sonar.server.es.ElasticsearchException: Fail to execute es requestES refresh request on indices ‘components’
caused by: java.lang.RuntimeException: Request cannot be executed; I/O reactor status : STOPPED
any advise please?
Thanks
Lalit KHATRI
es.log file show below error
2022.03.11 17:53:47 WARN es[][stderr] java.lang.reflect.InvocationTargetException
2022.03.11 17:53:47 WARN es[][stderr] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/java.lang.reflect.Method.invoke(Method.java:566)
2022.03.11 17:53:47 WARN es[][stderr] at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
2022.03.11 17:53:47 WARN es[][stderr] at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:535)
2022.03.11 17:53:47 WARN es[][stderr] Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.jdk.internal.org.objectweb.asm")
2022.03.11 17:53:47 WARN es[][stderr] at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/java.security.AccessController.checkPermission(AccessController.java:897)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1238)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:174)
2022.03.11 17:53:47 WARN es[][stderr] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
2022.03.11 17:53:47 WARN es[][stderr] at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71)
2022.03.11 17:53:47 WARN es[][stderr] at Log4jHotPatch.agentmain(Log4jHotPatch.java:93)
Hi @Rajesh_Mundhe,
It looks like you’re using a non-standard build of SonarQube. I suggest you try upgrading to 8.9.7 LTS or 9.3, the current version.
Ann
More information about this error
org.sonar.server.es.ElasticsearchException: Fail to execute es requestES refresh request on indices 'components'
at org.sonar.server.es.EsClient.execute(EsClient.java:313)
at org.sonar.server.es.EsClient.refresh(EsClient.java:155)
at org.sonar.server.es.BulkIndexer.stop(BulkIndexer.java:128)
at org.sonar.server.component.index.ComponentIndexer.delete(ComponentIndexer.java:170)
at org.sonar.ce.task.projectanalysis.purge.IndexPurgeListener.onComponentsDisabling(IndexPurgeListener.java:41)
at org.sonar.db.purge.PurgeDao.purgeDisabledComponents(PurgeDao.java:107)
at org.sonar.db.purge.PurgeDao.purge(PurgeDao.java:70)
at org.sonar.ce.task.projectanalysis.purge.ProjectCleaner.purge(ProjectCleaner.java:63)
at org.sonar.ce.task.projectanalysis.purge.PurgeDatastoresStep.execute(PurgeDatastoresStep.java:54)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: Request cannot be executed; I/O reactor status: STOPPED
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:904)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:276)
at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2699)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2137)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:2105)
at org.elasticsearch.client.IndicesClient.refresh(IndicesClient.java:900)
at org.sonar.server.es.EsClient.lambda$refresh$9(EsClient.java:155)
at org.sonar.server.es.EsClient.execute(EsClient.java:311)
... 27 common frames omitted
Caused by: java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED
at org.apache.http.util.Asserts.check(Asserts.java:46)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)
at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:284)
... 35 common frames omitted
Hi,
You may be using some variation on 9.3, but you’re not using the version we issued. The standard 9.3 distribution does not include
Ann
Thanks, @ganncamp, I am using the SonarQube developer edition. I will redeploy and cross-check again. I used https://binaries.sonarsource.com/CommercialDistribution/sonarqube-developer/sonarqube-developer-9.3.0.51899.zip link to download. I this correct version?
@ganncamp Issue with elasticsearch 7.16.3 version. This version has the Log4jHotPatch. This issue is throwing due to that. I think sonarQube should include the new elasticsearch version. I set the es Log level to TRACE in that logs it shows all details.
@ganncamp thanks for your help, it is resolved now. The issue was that the old version of Hot patch dependency was cached and considered.
@Rajesh_Mundhe and @ganncamp Could you help me on this?
I am using 9.2.3 version of Sonarqube and facing issue. Could you elaborate the resolution for the issue?
@prasad-clouduser the resolution is to upgrade to the official distribution of the latest version.
Ann
9.2.3 Community ain’t the official release?
Below are the complete error details:
org.sonar.server.es.ElasticsearchException: Fail to execute es requestES refresh request on indices 'components'
at org.sonar.server.es.EsClient.execute(EsClient.java:313)
at org.sonar.server.es.EsClient.refresh(EsClient.java:155)
at org.sonar.server.es.BulkIndexer.stop(BulkIndexer.java:128)
at org.sonar.server.component.index.ComponentIndexer.delete(ComponentIndexer.java:170)
at org.sonar.ce.task.projectanalysis.purge.IndexPurgeListener.onComponentsDisabling(IndexPurgeListener.java:41)
at org.sonar.db.purge.PurgeDao.purgeDisabledComponents(PurgeDao.java:107)
at org.sonar.db.purge.PurgeDao.purge(PurgeDao.java:70)
at org.sonar.ce.task.projectanalysis.purge.ProjectCleaner.purge(ProjectCleaner.java:63)
at org.sonar.ce.task.projectanalysis.purge.PurgeDatastoresStep.execute(PurgeDatastoresStep.java:54)
at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:81)
at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:72)
at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:59)
at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:81)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: Request cannot be executed; I/O reactor status: STOPPED
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:904)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:276)
at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2699)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2137)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:2105)
at org.elasticsearch.client.IndicesClient.refresh(IndicesClient.java:900)
at org.sonar.server.es.EsClient.lambda$refresh$9(EsClient.java:155)
at org.sonar.server.es.EsClient.execute(EsClient.java:311)
... 27 more
Caused by: java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED
at org.apache.http.util.Asserts.check(Asserts.java:46)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)
at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:284)
... 35 more
You seem to be getting a different error than the OP. Please create a new thread and provide all relevant details.
Ann
@ganncamp we are same issues on community edition 8.9.6 (Request cannot be executed; I/O reactor status: STOPPED). All our background tasks are failing due to I/O error. can you please guide us how to upgrade (hotpatch) elastick search version to 7.16.3. I don’t see proper instruction.
Hello @navaneeth.ramakrishn,
No, I cannot tell you how to hotpatch SonarQube. I can only tell you to upgrade to the latest point version, which already has all the patches you need.
HTH,
Ann
@ganncamp thanks for your response. we always used to upgrade LTS and Noticed on 8.9.7/8.9.8 LTS don’t have Elasticsearch 7.16.3 libs version. Attached screen shot. Is that something you recommend to upgrade 9.3 or above versions?
