Fail to execute es requestES refresh request on indices 'components'

shiva_sq · July 27, 2022, 1:08pm

Must-share information (formatted with Markdown):

which versions are you using (SonarQube, Scanner, Plugin, and any relevant extension)
SonarQube 9.5 latest, Scanner 4.7
what are you trying to achieve
Regular scan
what have you tried so far to achieve this

Error Details
org.sonar.server.es.ElasticsearchException: Fail to execute es requestES refresh request on indices 'components'
	at org.sonar.server.es.EsClient.execute(EsClient.java:313)
	at org.sonar.server.es.EsClient.refresh(EsClient.java:155)
	at org.sonar.server.es.BulkIndexer.stop(BulkIndexer.java:128)
	at org.sonar.server.component.index.ComponentIndexer.delete(ComponentIndexer.java:170)
	at org.sonar.ce.task.projectanalysis.purge.IndexPurgeListener.onComponentsDisabling(IndexPurgeListener.java:41)
	at org.sonar.db.purge.PurgeDao.purgeDisabledComponents(PurgeDao.java:107)
	at org.sonar.db.purge.PurgeDao.purge(PurgeDao.java:70)
	at org.sonar.ce.task.projectanalysis.purge.ProjectCleaner.purge(ProjectCleaner.java:63)
	at org.sonar.ce.task.projectanalysis.purge.PurgeDatastoresStep.execute(PurgeDatastoresStep.java:54)
	at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:80)
	at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:71)
	at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:58)
	at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
	at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
	at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
	at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
	at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: Request cannot be executed; I/O reactor status: STOPPED
	at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:926)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:299)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:287)
	at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2699)
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2137)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:2105)
	at org.elasticsearch.client.IndicesClient.refresh(IndicesClient.java:900)
	at org.sonar.server.es.EsClient.lambda$refresh$9(EsClient.java:155)
	at org.sonar.server.es.EsClient.execute(EsClient.java:311)
	... 27 more
Caused by: java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED
	at org.apache.http.util.Asserts.check(Asserts.java:46)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)
	at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:295)
	... 35 more

Please let me know how to fix this

Thanks
Shiva

Colin · July 27, 2022, 1:15pm

It seems like something has gone wrong with your Elasticsearch index.

I would recommend checking logs/es.log file of your SonarQube installation directory for additional information.

shiva_sq · July 27, 2022, 1:39pm

Do we need to check anything specific? in it
Need to share the log in private

shiva_sq · July 27, 2022, 3:27pm

2022.07.27 09:26:29 DEBUG es[i.n.u.i.l.InternalLoggerFactory] Using Log4J2 as the default logging framework
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent0] -Dio.netty.noUnsafe: true
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent0] sun.misc.Unsafe: unavailable (io.netty.noUnsafe)
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent0] Java version: 11
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent0] java.nio.DirectByteBuffer.(long, int): unavailable
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent] maxDirectMemory: 536870912 bytes (maybe)
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent] -Dio.netty.tmpdir: /opt/Sonar/sonarqube-9.5.0.56709/temp (java.io.tmpdir)
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent] -Dio.netty.bitMode: 64 (sun.arch.data.model)
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent] -Dio.netty.maxDirectMemory: -1 bytes
2022.07.27 09:26:29 DEBUG es[i.n.u.i.PlatformDependent] -Dio.netty.uninitializedArrayAllocationThreshold: -1
2022.07.27 09:26:29 DEBUG es[i.n.u.i.CleanerJava9] java.nio.ByteBuffer.cleaner(): unavailable
java.lang.UnsupportedOperationException: sun.misc.Unsafe unavailable
at io.netty.util.internal.CleanerJava9.(CleanerJava9.java:68) [netty-common-4.1.66.Final.jar:4.1.66.Final]
at io.netty.util.internal.PlatformDependent.(PlatformDependent.java:193) [netty-common-4.1.66.Final.jar:4.1.66.Final]
at io.netty.util.ConstantPool.(ConstantPool.java:34) [netty-common-4.1.66.Final.jar:4.1.66.Final]
at io.netty.util.AttributeKey$1.(AttributeKey.java:27) [netty-common-4.1.66.Final.jar:4.1.66.Final]
at io.netty.util.AttributeKey.(AttributeKey.java:27) [netty-common-4.1.66.Final.jar:4.1.66.Final]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.(Netty4HttpServerTransport.java:294) [transport-netty4-client-7.17.1.jar:7.17.1]
at org.elasticsearch.transport.Netty4Plugin.getSettings(Netty4Plugin.java:45) [transport-netty4-client-7.17.1.jar:7.17.1]
at org.elasticsearch.plugins.PluginsService.lambda$getPluginSettings$0(PluginsService.java:84) [elasticsearch-7.17.1.jar:7.17.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) [?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) [?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?]

These are some logs from es.log

shiva_sq · July 28, 2022, 1:10pm

Can anyone help me on this please let me know.
I got the same error today. I tried to modify “sonar.es.javaOpts=-Xmx4g -Xms4g -XX:+HeapDumpOnOutOfMemoryError”
It did not work.

Not sure how to fix this.

shiva_sq · July 28, 2022, 1:41pm

[spoiler]
[details="Summary"]
Status	Task	ID	Submitter		Submitted	Started	Finished	Duration
FAILED	V12_SECURITY/SYS 	AYJEI9N8FqzeVV45UZTd	admin	28-Jul-22	5:29:35 AM	7:43:06 AM	7:43:37 AM	31s
FAILED	V12_SECURITY/VERSION_SCRIPTS 	AYJEI3MmFqzeVV45UZTc	admin		5:29:10 AM	7:41:51 AM	7:43:05 AM	1min 14s
FAILED	V12_SECURITY/PM 	AYJEIw3xFqzeVV45UZTb	admin		5:28:44 AM	7:41:21 AM	7:41:51 AM	30s
FAILED	V12_SECURITY/CMIC 	AYJEIqyPFqzeVV45UZTa	admin		5:28:19 AM	7:40:50 AM	7:41:20 AM	30s
FAILED	V12_SECURITY/BI 	AYJEIk8PFqzeVV45UZTZ	admin		5:27:55 AM	7:40:22 AM	7:40:50 AM	27s
FAILED	V12_SECURITY/PY 	AYJEIfDpFqzeVV45UZTY	admin		5:27:31 AM	7:39:55 AM	7:40:22 AM	26s
FAILED	V12_SECURITY/SD 	AYJEIZVFFqzeVV45UZTX	admin		5:27:08 AM	7:39:26 AM	7:39:55 AM	29s
FAILED	V12_SECURITY/FORMS 	AYJEITORFqzeVV45UZTW	admin		5:26:42 AM	7:39:00 AM	7:39:25 AM	25s
FAILED	V12_SECURITY/SC 	AYJEINkxFqzeVV45UZTV	admin		5:26:19 AM	7:38:33 AM	7:38:59 AM	26s
FAILED	V12_SECURITY/MISCELLANEOUS 	AYJEIH2bFqzeVV45UZTU	admin		5:25:56 AM	7:37:53 AM	7:38:33 AM	39s
FAILED	V12_SECURITY/TM 	AYJEIB3yFqzeVV45UZTT	admin		5:25:31 AM	7:37:26 AM	7:37:53 AM	27s
FAILED	V12/JC 	AYJEH6t4FqzeVV45UZTS	admin		5:25:07 AM	7:32:39 AM	7:37:26 AM	4min 46s
FAILED	V12/JB 	AYJEHvj4FqzeVV45UZTR	admin		5:24:19 AM	7:30:42 AM	7:32:39 AM	1min 56s
FAILED	V12/BI 	AYJEHl1EFqzeVV45UZTQ	admin		5:23:43 AM	7:28:58 AM	7:30:42 AM	1min 44s
FAILED	V12/WKF 	AYJEHZslFqzeVV45UZTP	admin		5:22:48 AM	7:27:32 AM	7:28:57 AM	1min 25s
FAILED	V12/FORMS 	AYJEHSgwFqzeVV45UZTO	admin		5:22:17 AM	7:27:07 AM	7:27:32 AM	25s
FAILED	V12/FA 	AYJEHMqvFqzeVV45UZTN	admin		5:21:54 AM	7:26:23 AM	7:27:06 AM	43s
FAILED	V12/WO 	AYJEHGbKFqzeVV45UZTM	admin		5:21:28 AM	7:25:57 AM	7:26:23 AM	25s
FAILED	V12/BD 	AYJEHAmzFqzeVV45UZTL	admin		5:21:04 AM	7:25:10 AM	7:25:56 AM	45s
FAILED	V12/MS 	AYJEG6IsFqzeVV45UZTK	admin		5:20:39 AM	7:23:48 AM	7:25:10 AM	1min 21s
FAILED	V12/IMG 	AYJEGypdFqzeVV45UZTJ	admin		5:20:08 AM	7:22:09 AM	7:23:48 AM	1min 39s
FAILED	V12/OM 	AYJEGrPNFqzeVV45UZTI	admin		5:19:37 AM	7:20:45 AM	7:22:08 AM	1min 23s
FAILED	V12/GT 	AYJEGkKkFqzeVV45UZTH	admin		5:19:08 AM	7:19:53 AM	7:20:44 AM	51s
FAILED	V12/UE 	AYJEGdpIFqzeVV45UZTG	admin		5:18:41 AM	7:19:07 AM	7:19:53 AM	45s
FAILED	V12/SD 	AYJEGUsJFqzeVV45UZTF	admin		5:18:15 AM	7:16:20 AM	7:19:07 AM	2min 47s

[/details]
[/spoiler]

Colin · July 29, 2022, 9:05am

I would suggest you try clearing your Elasticsearch index and letting it rebuild.

Stop your SonarQube instance
Delete the data/es7 folder from your SonarQube installation directory
Start your SonarQube instance

shiva_sq · July 29, 2022, 11:30am

I tried it already
But the issue is not resolved yet

Still projects are failing to publish on SonarQube

Java heapspace issue
I tried with modifying my to -Xmx=4g

No luck with that
Thinking of going with 8g now.
Please let me know if it solve my issue

Thanks

shiva_sq · July 29, 2022, 11:41am

sonar.es.javaOpts=-Xmx8g -Xms8g -XX:+HeapDumpOnOutOfMemoryError
sonar.search.javaOpts=-Xmx8g -Xms8g -XX:MaxDirectMemorySize=2g -XX:+HeapDumpOnOutOfMemoryError

even this one did not work

Colin · July 29, 2022, 6:32pm

It looks like the OutOfMemory is coming from the ComputeEngine, so I would suggest adjusting sonar.ce.javaOpts – be careful that you aren’t allocating more memory than is actually available on your system.

Topic		Replies	Views
Fail to execute es requestES refresh request on indices 'components' SonarQube Server / Community Build sonarqube , quality-gate	4	1180	July 24, 2024
Fail to execute es requestES refresh request on indices 'components' SonarQube Server / Community Build	5	57	August 21, 2025
org.sonar.server.es.ElasticsearchException SonarQube Server / Community Build	2	150	July 26, 2024
Sonarqube throwing Elasticsearch exception SonarQube Server / Community Build sales_team_notified	1	109	February 11, 2025
Intermittent errors SonarQube Server / Community Build	3	199	April 11, 2024

Fail to execute es requestES refresh request on indices 'components'

Related topics