Java::JavaLang::OutOfMemoryError: GC overhead limit exceeded


(sunil kumar) #1

Continously receiving the OutOfMemoryError in our environment .Need an help to resolve it.
Below is stack trace
ava::JavaLang::OutOfMemoryError: GC overhead limit exceeded: SELECT xxxx ORDER BY xxxx
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract_adapter.rb:227:in log' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract_adapter.rb:212:inlog’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-jdbc-adapter-1.1.3/lib/arjdbc/jdbc/adapter.rb:183:in execute' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-jdbc-adapter-1.1.3/lib/arjdbc/jdbc/adapter.rb:275:inselect’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-jdbc-adapter-1.1.3/lib/arjdbc/jdbc/adapter.rb:202:in jdbc_select_all' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract/query_cache.rb:60:insele
ct_all_with_query_cache’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract/query_cache.rb:81:in cach e_sql' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract/query_cache.rb:60:insele
ct_all_with_query_cache’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/base.rb:665:in find_by_sql' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/base.rb:1582:infind_every’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/base.rb:619:in find' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/base.rb:639:inall’
/opt/sonarqube/web/WEB-INF/app/controllers/api/components_controller.rb:35:in suggestions' org/jruby/RubyKernel.java:2223:insend’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/base.rb:1333:in perform_action' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb:617:incall_filters’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb:610:in perform_action_with_filters' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/benchmarking.rb:68:inperform_action_with_benchmark’
/opt/sonarqube/web/WEB-INF/gems/gems/activesupport-2.3.15/lib/active_support/core_ext/benchmark.rb:17:in ms' jar:file:/opt/sonarqube/lib/server/jruby-complete-1.7.9.jar!/META-INF/jruby.home/lib/ruby/1.8/benchmark.rb:308:inrealtime’
/opt/sonarqube/web/WEB-INF/gems/gems/activesupport-2.3.15/lib/active_support/core_ext/benchmark.rb:17:in ms' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/benchmarking.rb:68:inperform_action_with_benchmark’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/rescue.rb:160:in perform_action_with_rescue' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/flash.rb:151:inperform_action_with_flash’
org/jruby/RubyKernel.java:2223:in send' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/base.rb:532:inprocess’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb:606:in process_with_filters' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/base.rb:391:inprocess’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/base.rb:386:in call' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/routing/route_set.rb:450:incall’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/dispatcher.rb:87:in dispatch' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/dispatcher.rb:85:indispatch’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/dispatcher.rb:121:in _call' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/dispatcher.rb:130:inbuild_middleware_stack’
org/jruby/RubyProc.java:290:in call' org/jruby/RubyProc.java:224:incall’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/query_cache.rb:29:in call' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract/query_cache.rb:34:incach
e’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/query_cache.rb:9:in cache' /opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/query_cache.rb:28:incall’
/opt/sonarqube/web/WEB-INF/gems/gems/activerecord-2.3.15/lib/active_record/connection_adapters/abstract/connection_pool.rb:361:in
call' /opt/sonarqube/web/WEB-INF/config/environment.rb:67:incall’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/string_coercion.rb:25:in call' /opt/sonarqube/web/WEB-INF/gems/gems/rack-1.1.6/lib/rack/head.rb:9:incall’
/opt/sonarqube/web/WEB-INF/gems/gems/rack-1.1.6/lib/rack/methodoverride.rb:24:in call' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/params_parser.rb:15:incall’
file:/opt/sonarqube/lib/server/jruby-rack-1.1.13.2.jar!/jruby/rack/session_store.rb:70:in context' /opt/sonarqube/web/WEB-INF/gems/gems/rack-1.1.6/lib/rack/session/abstract/id.rb:58:incall’
/opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/failsafe.rb:26:in call' /opt/sonarqube/web/WEB-INF/gems/gems/actionpack-2.3.15/lib/action_controller/dispatcher.rb:106:incall’
file:/opt/sonarqube/lib/server/jruby-rack-1.1.13.2.jar!/rack/adapter/rails.rb:34:in serve_rails' file:/opt/sonarqube/lib/server/jruby-rack-1.1.13.2.jar!/rack/adapter/rails.rb:39:incall’
file:/opt/sonarqube/lib/server/jruby-rack-1.1.13.2.jar!/rack/handler/servlet.rb:22:in `call’


(Julien Lancelot) #2

Hi @ksunil,

From the logs you’ve reported, I can see that the SonarQube version you’re using is pretty old, as there is some rails logs, and rails has been removed in SonarQube 6.3.

I highly recommend you to upgrade to latest LTS 6.7.

Regards,
Julien Lancelot


(sunil kumar) #3

Thanks Julien for the information provided.Could you help on the below analysis of heap dump shows two suspects from MAT
1)
The thread org.apache.tomcat.util.threads.TaskThread @ 0x81c8e310 http-nio-0.0.0.0-9000-exec-1 keeps local variables with total size 1,238,624,320 (74.47%) bytes.

The memory is accumulated in one instance of “java.lang.Object[]” loaded by “<system class loader>” .

One instance of “com.mysql.jdbc.JDBC4ResultSet” loaded by “sun.misc.Launcher$AppClassLoader @ 0x80037a00” occupies 345,115,680 (20.75%) bytes. The memory is accumulated in one instance of “java.lang.Object[]” loaded by “<system class loader>” .

Keywords
java.lang.Object
com.mysql.jdbc.JDBC4ResultSet
sun.misc.Launcher$AppClassLoader @ 0x80037a00