Java application "java.lang.OutOfMemoryError: GC overhead limit exceeded"

Environment

• Java
• OpenJDK
• Oracle JDK

Issue

• The following error appears in the server log (e.g. JBoss server.log):

ERROR [STDERR] java.lang.OutOfMemoryError: GC overhead limit exceeded

• We would like to root cause analysis of the heap dump generated by one of our application (GRCC) and would like to get some recommendation on heap size parameter setting.

Resolution

• If the new generation size is explicitly defined with JVM options, decrease the size or remove the relevant JVM options entirely to unconstrain the JVM and provide more space in the old generation for long lived objects.
• If there is unintended object retention, typically code and/or configuration changes are needed.
• If the retention looks normal, and it is a load issue, the heap size would need to be increased.

Root Cause

• This error is thrown if more than 98% of the total time is spent doing garbage collection and less than 2% of the heap is recovered for 5 consecutive gc cycles. It is intended to prevent applications from running for an extended period of time while making little or no progress reclaiming objects (e.g. when the heap is too small, there is a memory leak, or the old generation is disproportionately small compared to the new generation). It is a throttle to prevent the JVM from swamping the environment it is running in. Disabling this feature with the -XX:-UseGCOverheadLimit JVM option is not recommended.
• Increased memory usage during JBoss EAP buddy replication state transfer
• JBoss - Java heap retention in FileCleaningTracker$Tracker
• We are seeing Hibernate store a lot of information in the Session (first level cache) and eventually see OutOfMemory exceptions
• JON Agent JVM consuming a lot of memory due to too many bindings set configured in bindings-jboss-beans.xml
• JBoss - Java heap retention in java.lang.ref.Finalizers
• Fuse Fabric console goes OOME because of accumulating org.fusesource.fabric.jaas.ZookeeperProperties
• OutOfMemory GC Overhead using Oracle JDBC Driver
• This content is not included.Satellite WebUI 'java.lang.OutOfMemoryError: GC overhead limit exceeded'
• Java heap retention in JceSecurity's static verificationResults map
• Slowness / Memory issues when starting JBoss with many JSF 2 apps on java 8 where "ZipFileInflaterInputStream" count is growing and pending finalization
• JBoss reaching OOME during start up after update to EAP 7.4 Update 11
• JBoss EAP domain controller reaches OOME with large org.jboss.as.protocol.mgmt.ActiveOperationImpl overhead

Diagnostic Steps

• Check the JVM options to see if the young generation size is being explicitly set (e.g. -XX:NewSize, -XX:MaxNewSize, -Xmx).

• Enable the standard garbage collection logging options and analyze the logging. Check the ratio of the old/new generations and note the amount of space reclaimed in the new and old generations when the issue happens.

  • How do I enable Java garbage collection logging?
  • How do I analyze Java garbage collection logging?

• Get a heap dump as close to OutOfMemoryError as possible, and analyze it  to determine where the retention is:

  • How do I create a Java heap dump?
  • How do I analyze a Java heap dump?

• Note: Sometimes this type of OutOfMemoryError does not result in a heap dump when the -XX:+HeapDumpOnOutOfMemoryError option is used. In those cases it will be necessary to get a heap dump manually.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.