Fail to execute es requestES refresh request on indices 'components'

We are experiencing recurring issues with SonarQube where an ElasticsearchException is thrown, specifically when executing ES refresh requests on indices ‘components’. Below is the detailed error log:

org.sonar.server.es.ElasticsearchException: Fail to execute es requestES refresh request on indices 'components'
        at org.sonar.server.es.EsClient.execute(EsClient.java:313)
        at org.sonar.server.es.EsClient.refresh(EsClient.java:155)
        at org.sonar.server.es.BulkIndexer.stop(BulkIndexer.java:128)
        at org.sonar.server.component.index.ComponentIndexer.delete(ComponentIndexer.java:164)
        at org.sonar.ce.task.projectanalysis.purge.IndexPurgeListener.onComponentsDisabling(IndexPurgeListener.java:41)
        at org.sonar.db.purge.PurgeDao.purgeDisabledComponents(PurgeDao.java:106)
        at org.sonar.db.purge.PurgeDao.purge(PurgeDao.java:69)
        at org.sonar.ce.task.projectanalysis.purge.ProjectCleaner.purge(ProjectCleaner.java:63)
        at org.sonar.ce.task.projectanalysis.purge.PurgeDatastoresStep.execute(PurgeDatastoresStep.java:54)
        at org.sonar.ce.task.step.ComputationStepExecutor.executeStep(ComputationStepExecutor.java:79)
        at org.sonar.ce.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:70)
        at org.sonar.ce.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:57)
        at org.sonar.ce.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:75)
        at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.executeTask(CeWorkerImpl.java:212)
        at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.run(CeWorkerImpl.java:194)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:160)
        at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.RuntimeException: Request cannot be executed; I/O reactor status: STOPPED
        at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:940)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:300)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
        at org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2699)
        at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:2171)
        at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:2137)
        at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:2105)
        at org.elasticsearch.client.IndicesClient.refresh(IndicesClient.java:900)
        at org.sonar.server.es.EsClient.lambda$refresh$9(EsClient.java:155)
        at org.sonar.server.es.EsClient.execute(EsClient.java:311)
        ... 27 common frames omitted
Caused by: java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED
        at org.apache.http.util.Asserts.check(Asserts.java:46)
        at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)
        at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296)
        ... 35 common frames omitted
  • We are using Sonar version - Developer EditionVersion 9.9 (build 65466)
  • Sonar scanner version - SonarScanner 4.8.0.2856
  • SonarQube deployed using zip.
  • We tried the following work arounds,
 Initially, we restarted the SonarQube server, which resolved the issue temporarily.

Upon encountering the issue again, we followed the steps in the SonarQube community thread `https://community.sonarsource.com/t/fail-to-execute-es-requestes-refresh-request-on-indices-components/68817`:
Stopped the SonarQube server.
Cleared the $SONAR_HOME/data/es7 path.
Restarted the SonarQube server.
This resolved the issue temporarily once again.

After performing the restart, we noticed several .hprof files had been created, though it is unclear if they were generated as a result of our actions.

We need assistance in permanently resolving this issue. Please provide guidance or potential solutions to prevent this error from recurring.

Hey there.

Do you see any specific errors in the logs? Particularly es.log is going to be interesting if the issue is with Elasticsearch.

es.log (5.9 KB)
ce.log (6.2 KB)
I have attached both es.log and ce.log files.
There are no es.log found at the incident time.

It looks like you’re facing multiple errors. I suggest focusing on this one:

2024.07.05 13:22:27 ERROR ce[AZCB4Kksvq986wGbAhnv][o.s.c.t.CeWorkerImpl] Failed to execute task AZCB4Kksvq986wGbAhnv
java.lang.OutOfMemoryError: Java heap space

I suggest bumping up the maximum heap size of your Compute Engine. If you have a particularly large project, this can be necessary.

In your conf/sonar.properties file, you can adjust sonar.ce.javaOpts. I suggest a value of 2GB, assuming you have sufficient memory on the server.

sonar.ce.javaOpts=-Xmx2G -Xms128m -XX:+HeapDumpOnOutOfMemoryError

You’ll need to reboot the service for the changes to take affect (./sonar.sh restart)

Thank you so much for your assistance. The solution you provided is working and our issue is now resolved. I really appreciate your prompt and effective help.