How do I capture hs_err_pid.log in container environments?

Solution Verified - Updated

Environment

  • Red Hat Openshift Container Platform
  • Red Hat build of OpenJDK
    • 8u312+
    • 11.0.7+
    • 17

Issue

  • How do I save hs_err_pid.log in container environments?
  • The local file system is ephemeral in container environments. Do I have to attach a Persistent Volume to java container to save hs_err_pid.log?

Resolution

It is possible to set jvm option -XX:+ErrorFileToStdout to output hs_err_pid.log to stdout.
-XX:+ErrorFileToStdout is introduced in OpenJDK11 by Content from bugs.openjdk.java.net is not included.JDK-8220786: Create new switch to redirect error reporting output to stdout or stderr.

For a JBoss EAP container, add it to the environment variable JAVA_OPTS_APPEND:

oc set env dc/<deplocyment-config-name> JAVA_OPTS_APPEND=-XX:+ErrorFileToStdout

In case java process crashes, you can check the crash log via oc logs command as follows:

oc logs <application-pod-name>
...
08:40:04,609 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.3.1.GA (WildFly Core 10.1.7.Final-redhat-00001) started in 14388ms - Started 567 of 832 services (539 services are lazy, passive or on-demand)
08:40:04,613 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
08:40:04,613 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0054: Admin console is not enabled
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f66986c07cb (sent by kill), pid=866, tid=866
#
# JRE version: OpenJDK Runtime Environment (11.0.7+10) (build 11.0.7+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.7+10-LTS, mixed mode, sharing, tiered, compressed oops, parallel gc, linux-amd64)
# Problematic frame:
# C  [libpthread.so.0+0x97cb]  __GI___pthread_timedjoin_ex+0x18b
#
# Core dump will be written. Default location: /home/jboss/core.866
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
...
CPU:total 2 (initial active 1) (1 cores per cpu, 1 threads per core) family 6 model 42 stepping 1, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, aes, clmul, tsc
CPU Model and flags from /proc/cpuinfo:
model name	: Intel Xeon E312xx (Sandy Bridge)
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm xsaveopt arat

Memory: 4k page, physical 1048576k(689292k free), swap 0k(0k free)

vm_info: OpenJDK 64-Bit Server VM (11.0.7+10-LTS) for linux-amd64 JRE (11.0.7+10-LTS), built on Apr 15 2020 12:25:53 by "mockbuild" with gcc 8.3.1 20190507 (Red Hat 8.3.1-4)

END.
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
*** JBossAS process (866) received ABRT signal ***
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.