Java crash in libjvm.so due to LargePages

Solution Unverified - Updated

Environment

  • OpenJDK 1.8

Issue

  • Fatal error log shows the following:
...
# Problematic frame:
# V  [libjvm.so+0x906001]
...

---------------  T H R E A D  ---------------

Current thread (0x00007f9a6c098000):  VMThread [stack: 0x00007f9a3ab68000,0x00007f9a3ac69000] [id=27694]

siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000

...
  • xsos output shows nearly all of system memory assigned to HugePages:

MEMORY
  Stats graphed as percent of MemTotal:
    MemUsed    ▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊  99.8%
    Buffers    ..................................................   0.0%
    Cached     ..................................................   0.2%
    HugePages  ▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊▊.  98.5%
    Dirty      ..................................................   0.0%
  RAM:
    62.9 GiB total ram
    62.8 GiB (100%) used
    62.6 GiB (100%) used excluding Buffers/Cached
    0.01 GiB (0%) dirty
  HugePages:
    62 GiB pre-allocated to HugePages (99% of total ram)
    0 GiB of HugePages (0%) in-use by applications
  LowMem/Slab/PageTables/Shmem:
    0.1 GiB (0%) of total ram used for Slab
    0.01 GiB (0%) of total ram used for PageTables
    0 GiB (0%) of total ram used for Shmem
  Swap:

Resolution

All are possible solutions:

  • Assign a smaller percent of total memory to HugePages.
  • Remove XX:+UseLargePages.

Root Cause

HugePages can only be used by programs specifically designed to take advantage of them. They cannot be used by just any application, nor can they be used by the OS. Furthermore, applications that were designed to be able to use them must be configured to do so after making the HugePage reservation from the OS. For those reasons, it can be dangerous to have too large of a HugePage reservation. Out of memory errors and even the OS failing to boot are common problems with a too large a reservation (e.g. more than available RAM).

Diagnostic Steps

Check to see if the HugePage reservation and LargePages value is sufficiently less than available physical memory.

SBR
Product(s)
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.