Java high cpu due to continuous G1 concurrent cycles initiated by humongous allocations

Solution Unverified - Updated

Environment

  • OpenJDK
  • G1 garbage collector

Issue

  • Java high cpu due to continuous G1 concurrent cycles initiated by humongous allocations.

Resolution

Increase the G1HeapRegionSize. For example:

-XX:G1HeapRegionSize=16m

Root Cause

Objects larger than 1/2 x G1HeapRegionSize are treated as humongous and allocated directly in the old generation into a contiguous set of humongous regions.

Every humongous allocation triggers an Initiating Heap Occupancy Percent (IHOP) check. The old generation occupancy as a percent of the total heap size is calculated, and if it reaches the IHOP threshold, a concurrent cycle is initiated to reclaim space.

G1HeapRegionSize is a size that is a power of 2m, up to a max of 32m. The default value is based on max heap:

 <4GB: 1MB
 <8GB: 2MB
<16GB: 4MB
<32GB: 8MB
<64GB: 16MB
64GB+: 32MB

High allocation rates of humongous objects could result in continuous concurrent cycles one after another, contributing to high cpu.

Diagnostic Steps

  • Check standard high cpu data and verify the high cpu is caused by a G1 Conc thread. For example:
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
1234567 jbossas   20   0 6425636   4.3g  40160 R  94.1  28.3  48:41.42 G1 Conc#0
  • Check gc logging to see if humongous allocations are initiating G1 concurrent cycles one after another:
$ grep -E "(Humongous Allocation\) |Concurrent Cycle)" gc.log

[14293ms] GC(107) Pause Young (Concurrent Start) (G1 Humongous Allocation) 2791M->2747M(2934M) 25.148ms
[14293ms] GC(108) Concurrent Cycle
[29842ms] GC(108) Concurrent Cycle 15548.571ms
[31128ms] GC(138) Pause Young (Concurrent Start) (G1 Humongous Allocation) 2820M->2766M(2934M) 20.221ms
[31128ms] GC(139) Concurrent Cycle
[47383ms] GC(139) Concurrent Cycle 16254.332ms
[48262ms] GC(171) Pause Young (Concurrent Start) (G1 Humongous Allocation) 2825M->2795M(2966M) 24.403ms
[48262ms] GC(172) Concurrent Cycle
[66417ms] GC(172) Concurrent Cycle 18154.897ms
[67363ms] GC(201) Pause Young (Concurrent Start) (G1 Humongous Allocation) 2799M->2786M(2995M) 37.344ms
[67364ms] GC(202) Concurrent Cycle
[84411ms] GC(202) Concurrent Cycle 17047.453ms
SBR
Product(s)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.