Elasticsearch error - all shards failed - CircuitBreakingException

SonarQube
,
1

hello,
I have a problem with my sonar environment.
when I click on a certain page it returns me the following error

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed; shardFailures {[5qjZTWBXQhKDywnBGEzTlA][issues][0]: ElasticsearchException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: CircuitBreakingException[[FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; }{[5qjZTWBXQhKDywnBGEzTlA][issues][1]: ElasticsearchException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: CircuitBreakingException[[FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; }{[5qjZTWBXQhKDywnBGEzTlA][issues][2]: ElasticsearchException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: CircuitBreakingException[[FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; }{[5qjZTWBXQhKDywnBGEzTlA][issues][3]: ElasticsearchException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: CircuitBreakingException[[FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; }{[5qjZTWBXQhKDywnBGEzTlA][issues][4]: ElasticsearchException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; nested: CircuitBreakingException[[FIELDDATA] Data too large, data for [status] would be larger than limit of [633785548/604.4mb]]; }
	at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:237) ~[elasticsearch-1.7.2.jar:na]
	at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$1.onFailure(TransportSearchTypeAction.java:183) ~[elasticsearch-1.7.2.jar:na]
	at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:565) ~[elasticsearch-1.7.2.jar:na]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
	... 1 common frames omitted
2018.08.22 13:07:05 DEBUG web[http] GET /api/issues/search?facets=statuses&ps=1&additionalFields=_all&resolved=false&componentUuids=AWLjJtaMVaTD2TOpduMQ | time=465ms
2018.08.22 13:07:40 DEBUG web[http] GET /api/components/suggestions?s=op | time=63453ms
2

Hi,

It’s going to be difficult to help you without more information.

First, apparently chopped off the stacktrace. At the very least, we need the initial error line.

It would also be useful to know

  • your SQ version
  • exactly what the “certain page” is (i.e. what URL).

Ann

1 Like
3

Hi,

To complement Ann’s response, the error you report here is a quite specific one:

This is an ElasticSearch problem. ElasticSearch is under water because it doesn’t have enough resources at its disposal (reminder: ElasticSearch is a search engine used by SonarQube to index issues, rules etc. so that it can access this data rapidly without having to query the database all the time, see SonarQube Architecture ).

Most likely the bottom line here is that you’re running an instance which has grown over time, and that instance now doesn’t have enough resources at its disposal to perform all its operations while being responsive and stable.

It’s essential at this time to do a bit of monitoring i.e. keep track of CPU/memory consumption to detect early any bottleneck and adjust the configuration if needed. Memory configuration is deployment-specific, the only way to tell is to monitor and provision resources accordingly.

1 Like