Verify OpenJDK has debug symbols installed

Solution Verified - Updated 14 Jun 2024

Environment

Red Hat Enterprise Linux (RHEL)
- 8.x
- 7.x
- 6.x
OpenJDK:
- 8
- 11
Microsoft Windows Platform

Issue

How to list symbols from OpenJDK?
Does my JDK has symbols installed?

Resolution

The solution below shows how to install for several platforms and verify the installation.

Debug symbols Installation

To install debug symbols:

RHEL6:

$ debuginfo-install java-(1.6.0|1.7.0|1.8.0)-openjdk

RHEL7¹:

$ debuginfo-install java-(1.6.0|1.7.0|1.8.0|11)-openjdk

RHEL8²:

$ debuginfo-install java-(1.8.0|11)-openjdk
$ debuginfo-install java-(1.8.0|11)-openjdk-headless

This will install java-<version>-openjdk-debuginfo and java-<version>-openjdk-headless-debuginfo (RHEL8), extra packages that provide debug symbols for OpenJDK binaries. These packages are not self-sufficient and do not contain executable binaries.

Note: debuginfo-install is provided by the yum-utils package.

Microsoft Windows Platform:

Go to Customer Portal
Download the respective Debugging Symbols for Linux 64 Bit package. Example for This content is not included.JDK 11.0.13: OpenJDK 11.0.13.8 Debugging Symbols for Windows 64 Bit
Unzip and copy it to the same directory where corresponding binaries are, basically all to OpenJDK bin directory, meaning everything from debuginfo bundle goes into the JAVA_HOME/bin/, except jvm.pdb and jvm.map go into the JAVA_HOME/jre/bin/server/ (JDK8) or JAVA_HOME/bin/server/ (JDK11/17). That's because jvm.dll is always loaded manually from this location.

Confirming debug symbols installation

To verify if you have debug symbols installed, run gdb which java and it will tell where the symbols come from:

$ gdb `which java`
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-110.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/bin/java...Reading symbols from /usr/lib/debug/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/bin/java.debug...done.
done.
(gdb)

To test it if the symbols are correctly loaded:

    #First start a program to load the symbol, ie. attach the thread debugging to libthread_db enabled
    Starting program: /usr/bin/java -X
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib64/libthread_db.so.1".
    [New Thread 0x7ffff7fd6700 (LWP 12632)]
    #
    #Then run info func JavaMain
    (gdb)  info func JavaMain
    All functions matching regular expression "JavaMain":

    File /usr/src/debug/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/openjdk/jdk/src/share/bin/java.c:
    int JavaMain(void *);

Instead of info func JavaMain one can use ptype command with any class, example oop:

(gdb) ptype oop
type = class oopDesc {
  private:
    volatile markOop _mark;
    oopDesc::_metadata _metadata;
    static BarrierSet *_bs;
  public:
    markOop mark() const;

There is also the possibility to see if the package is installed:

#verify for package 1.8.0 debug info
$ yum list installed | grep 'java-1.8.0-openjdk-debuginfo'
Repository opera is listed more than once in the configuration
java-1.8.0-openjdk-debuginfo.x86_64     1:1.8.0.161-2.b14.el7      @rhel-7-workstation-debug-rpms

If the package is installed but one can't find the location, verify if the correct package version/java version was installed and if the debug symbols are correct in the location, see below.

Debug symbols location:

When using gdb command with which java, one can see where the debug symbols are located.
The symbols are installed, by default, at /usr/lib/debug/lib/jvm:

Reading symbols from /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/bin/java...Reading symbols from /usr/lib/debug/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/bin/java.debug...done.

Exploring the directory it possible to see all the debug versions of the libraries, which will include java, javac, and javah:

$cd /usr/lib/debug/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64-debug
$tree
├── java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64
│   ├── bin
│   │   ├── appletviewer.debug
│   │   ├── javac.debug
│   │   ├── java.debug
│   │   └── xjc.debug
│   ├── jre
│   │   ├── bin
│   │   │   ├── java.debug
│   │   │   └── unpack200.debug
│   │   └── lib
│   │       ├── amd64
│   │       │   ├── jli
│   │       │   │   └── libjli.so.debug
│   │       │   ├── libzip.so.debug
│   │       │   └── server
│   │       │       ├── libjsig.so.debug -> ../libjsig.so.debug
│   │       │       └── libjvm.so.debug
│   │       └── jexec.debug
│   └── lib
│       ├── amd64
│       │   ├── jli
│       │   │   └── libjli.so.debug
│       │   └── libjawt.so.debug
│       └── jexec.debug
└── java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64-debug
    ├── bin
    │   ├── appletviewer.debug

In the above output, some lines were suppressed for presentation purposes.

Debug symbols in hs_error file

The JVM crash file, brings the stack section, which presents some information from the stack.

However, in recent versions OpenJDK strip the stack, using strip -g, which means that all debug symbols were removed. So, the crash file hs_error will be incomplete without the debug symbols installed. Moreover, for installing it, see Java application down due to JVM crash.

Example of not stripped hs_error file:

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xb83d2a]  Unsafe_SetLong+0xda
j  sun.misc.Unsafe.putLong(Ljava/lang/Object;JJ)V+0
j  Crash.main([Ljava/lang/String;)V+8
v  ~StubRoutines::call_stub
V  [libjvm.so+0x6c0e65]  JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0xc85
V  [libjvm.so+0x73cc0d]  jni_invoke_static(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*) [clone .constprop.1]+0x31d
V  [libjvm.so+0x73fd16]  jni_CallStaticVoidMethod+0x186
C  [libjli.so+0x48a2]  JavaMain+0x472
C  [libpthread.so.0+0x9432]  start_thread+0xe2

Example of stripped hs_error file:

Stack: [0x00007ff7e1a44000,0x00007ff7e1b44000],  sp=0x00007ff7e1b42850,  free space=1018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xa7ecab]
j  sun.misc.Unsafe.putAddress(JJ)V+0
j  Crash.crash()V+5
j  Crash.main([Ljava/lang/String;)V+0
v  ~StubRoutines::call_stub
V  [libjvm.so+0x67133a]
V  [libjvm.so+0x682bca]
V  [libjvm.so+0x6968b6]
C  [libjli.so+0x3989]
C  [libpthread.so.0+0x7dd5]  start_thread+0xc5

Notes

To verify which java/symlink is being used, for instance, with alternatives:

$ sudo alternatives --config java
    There are 2 programs which provide 'java'.
     Selection    Command
    -----------------------------------------------
    *+ 1           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/bin/java)
    2           java-1.7.0-openjdk.x86_64 (/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.171-2.6.13.0.el7_4.x86_64/jre/bin/java)

This points to /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/bin/java

Using nm command can show if the libjvm.so has ELF data and text symbols but not debug symbols , example:
Content from linux.die.net is not included.nm Linux command to list symbols from object file in the libjvm.so:

    $nm /usr/lib/debug/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/amd64/server/libjvm.so.debug 
    0000000000f85138 d _ZZN13TemplateTable6dconstEiE3one
    0000000000f85144 d _ZZN13TemplateTable6fconstEiE3one
    0000000000f85140 d _ZZN13TemplateTable6fconstEiE3two
    0000000000b39b18 r _ZZN13TemplateTable7convertEvE6is_nan
    0000000000fa5354 b _ZZN13VM_HeapDumper13do_load_classEP5KlassE16class_serial_num
    0000000000f807f0 d _ZZN14AbstractICache15call_flush_stubEPhiE5magic
    0000000000f807e0 d _ZZN14AbstractICache16invalidate_rangeEPhiE9firstTime
    0000000000fa35b0 b _ZZN14Deoptimization16trap_action_nameEiE3buf
    0000000000fa35d0 b _ZZN14Deoptimization16trap_reason_nameEiE3buf
    0000000000fa5380 b _ZZN14KlassInfoHisto7str_fmtEiE3buf
    0000000000fa5360 b _ZZN14KlassInfoHisto8perc_fmtEiE3buf
    0000000000f83670 d _ZZN14MacroAssembler10pow_or_expEbiE3two
    0000000000b1e800 r _ZZN14MacroAssembler15corrected_idivqEP12RegisterImplE8min_long
    0000000000fa54e0 b _ZZN14SparsePRTEntry9cards_numEvE1s
    0000000000fa68e0 b _ZZN15CallbackInvoker22report_primitive_fieldE22jvmtiHeapReferenceKindP7oopDesciPhcE14reference_info
    0000000000fa6920 b _ZZN15CallbackInvoker41invoke_advanced_object_reference_callbackE22jvmtiHeapReferenceKindP7oopDescS2_iE14reference_info

The package This content is not included.java-1.6.0-openjdk-debuginfo is available in the repositories rhel-7-server-debug-rpms and rhel-7-server-optional-debug-rpms, for RHEL 7.
²: The package This content is not included.java-11-openjdk-headless is available in the repository rhel-8-for-x86_64-appstream-rpms, for RHEL 8.

SBR

JVM & Diagnostics

Product(s)

Red Hat build of OpenJDK

Components

Category

Learn more

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.