Tomcat shutdown fails with a JVM crash because there is insufficient memory

Environment

• Red Hat Enterprise Linux
• Red Hat JBoss Enterprise Web Server
• Tomcat

Issue

• We try to shutdown the RHEL tomcat service. It fails with a JVM crash and tomcat continues to run. The JVM crash occurs because of insufficient memory:

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate ### bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.

Resolution

• Set jvm options for your Tomcat through CATALINA_OPTS instead of JAVA_OPTS. Please refer to this solution for details.

Root Cause

• JAVA_OPTS are used by both the tomcat JVM and the JVM that is started and used to carry out the tomcat service shutdown as described by How to set the JVM options for Tomcat on Linux. If you use a large heap or have little system memory beyond your defined heap size, then setting the heap in JAVA_OPTS can result in the shutdown process failing with this JVM crash since the shutdown JVM can't also be created with a heap of the same size as the Tomcat JVM.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.