java.lang.OutOfMemoryError "Cannot create GC thread. Out of system resources"

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux(RHEL)

    • 6.x
    • 7.x

  • Red Hat JBoss Enterprise Application Platform (EAP)

    • 6.x
    • 7.x

Issue

  • A fatal error was detected by the Java Runtime Environment while the jboss application server was started using a normal user name 'jboss'.
  • The hs_err fatal error log file detected the following error:
# java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.
# 
#  Internal Error (gcTaskThread.cpp:38), pid=31003, tid=1084184896
#  Error: Cannot create GC thread. Out of system resources.
  • Or:
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (workgroup.cpp:99), pid=40464, tid=0x00007efc4fa25700

Resolution

  1. On 32-bit, free up address space and/or memory (e.g. decreasing -Xmx, decreasing -Xss).
  2. Increase the process ulimit

Root Cause

Diagnostic Steps

The diagnostic steps are explained below:

  1. Review the fatal error log to see the number of threads is reasonable. If there are many more threads than expected, it could be a thread leak. If the thread count seems reasonable, it's likely the JVM process is hitting an OS limit.

  2. Create a sosreport around the time of the issue:

  • Determine the user and process id(s) the Java application causing the fatal error log by inspecting the output from the following:

    grep "java" sosreport/ps

  • Open file limit

    • Determine the open file limit for the user by inspecting <sosreport>/etc/security/limits.conf:

      grep "nofile" <sosreport>/etc/security/limits.conf
JAVA_USER    soft    nofile           1024
JAVA_USER    hard    nofile           8192

    • Determine the number of open files for the Java process(es) that caused the fatal error log to be created:

      grep "<JAVA_PID>" sosreport/sos_commands/process/lsof_-b_M_-n_-l | wc -l
1988

  • Max user processes

    • Determine the max user processes limit by inspecting <sosreport>. See What determines the default value of nproc(ulimit -u) on Red Hat Enterprise Linux?.

    • Determine the number of threads for JAVA_USER:

      grep "JAVA_USER" sosreport/sos_commands/process/ps_auxwwwm | wc -l
499

      Note: This will result in a slightly larger number than the actual number of threads due to the nature of ps_auxwwwm output, where every process has a summary entry. Even a process with a single thread has 2 entries in ps_auxwwwm.

  • Max number of threads

    • Determine the max number of threads per process limit:

      grep "kernel.pid_max" sosreport/sos_commands/kernel/sysctl_-a 
kernel.pid_max = 32768

    • Count the number of threads in the fatal error log.

  1. To do the investigation to tell what ulimit is being impacted, check the following:

  • number open files in /etc/security/limits.conf:

    soft    nofile           1024
hard    nofile           8192

    or both hard and soft could be set to the same value:

    - nofile 2048

    Compare ulimit -n and lsof output to see if the number of Java application sockets is exceeding the open file limit.

    For example:

    ulimit -n
lsof -p <JAVA_PID>

  • max user processes in /etc/security/limits.conf (RHEL5 and before) or /etc/security/limits.d/90-nproc.conf (RHEL6 on):

    soft    nproc           2048
hard    nproc           8192

    To see if the max user processes limit is being exceeded, compare ulimit -u output (run as the user running JBoss) to the number of threads in a thread dump to see if the max user processes limit is being exceeded.

    Compare ps -eLf output for the user running the Java process to see if the max user processes are being exceeded.

    For example (user = 'jbossas'): 

    ps -eLf | grep "^jbossas" | wc -l

  • max number of threads in /etc/sysctl.conf:

    kernel.pid_max = 65536

    To see if it is max number of threads, download the attached ThreadTest class (threadtest.java.zip) and compile it (javac ThreadTest.java) on the box running the Java application. Then run it as the user that the Java application is run under, passing in values such as 1500, 3000, 5000, and 10,000 and using the same JVM options used to start the Java application. Does the script complete? If not, what number does it stop on?

    For example:

    java -server -Xss128k -Xms1024M -Xmx1024M -XX:PermSize=128M -XX:MaxPermSize=128M -XX:+UseParallelGC -XX:ParallelGCThreads=4 -XX:+UseParallelOldGC -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 ThreadTest 5000
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.