JCMD usage in OpenJDK for troubleshooting

Solution Verified - Updated

Environment

  • OpenJDK 8
  • OpenJDK 11
  • OpenJDK 17
  • OpenJDK 21

Issue

  • How to use jcmd?
  • What options are available for jcmd?

Resolution

Jcmd is a tool to assist in memory management investigation by sending command requests to a running Java Virtual Machine.
Jcmd has a series of options to get an overview of the memory, and must be used in the same box that the JVM is running, i.e can not be used remotely and must be used with the same user that is executing . Also to read/write data the JVM must create a file called /tmp/.java_pid.

Command line usage

All the commands below are done with: jcmd $PID.

  1. To list process in the JVM, use jcmd without arguments or jcmd with -l:

    $ jcmd
    17085 sun.tools.jcmd.JCmd
    26670 /home/jboss-eap-7.2//jboss-modules.jar -mp /home/fdemeloj/jboss-eap-7.2//modules org.jboss.as.standalone -Djboss.home.dir=/home/fdemeloj/jboss-eap-7.2/ -Djboss.server.base.dir=/home/jboss-eap-7.2//standalone
    #Example with jcmd -l:
    $ jcmd -l
    16820 sun.tools.jcmd.JCmd -l
    26670 /jboss-eap-7.2//jboss-modules.jar -mp /home/fdemeloj/jboss-eap-7.2//modules org.jboss.as.standalone -Djboss.home.dir=/home/fdemeloj/jboss-eap-7.2/ -Djboss.server.base.dir=/home/jboss-eap-7.2//standalone
    
  2. To make sure the program has correct JVM flags, you can do VM.flags

    $ jcmd 26670 VM.flags
    26670:
    -XX:CICompilerCount=4 -XX:GCLogFileSize=3145728 -XX:InitialHeapSize=1367343104 -XX:MaxHeapSize=1367343104 -XX:MaxMetaspaceSize=268435456 -XX:MaxNewSize=455606272 -XX:MetaspaceSize=100663296 -XX:MinHeapDeltaBytes=524288 -XX:NativeMemoryTracking=detail -XX:NewSize=455606272 -XX:NumberOfGCLogFiles=5 -XX:OldSize=911736832 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:-TraceClassUnloading -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseGCLogFileRotation -XX:+UseParallelGC 
    
  3. And to list all options available of jcmd, use help, since there is no autocomplete this helpful (The list will only contain the possible commands to be used):

    $ jcmd 26670 help
    26670:
    The following commands are available:
    VM.native_memory
    ManagementAgent.stop
    ManagementAgent.start_local
    ManagementAgent.start
    GC.rotate_log
    Thread.print
    GC.class_stats
    GC.class_histogram
    GC.heap_dump
    GC.run_finalization
    GC.run
    VM.uptime
    VM.flags
    VM.system_properties
    VM.command_line
    VM.version
    help
    
  4. To get details about the JVM version:

    $ jcmd 26670 VM.version
    26670:
    OpenJDK 64-Bit Server VM version 25.161-b14
    JDK 8.0_161
    
  5. System properties VM.system_properties

    $ jcmd 26670 VM.system_properties
    26670:
    #Fri Sep 25 14:36:13 EDT 2020
    jboss.qualified.host.name=fdemeloj.yyz.csb
    java.vendor=Oracle Corporation
    logging.configuration=file\:/home/fdemeloj/jboss-eap-7.2//standalone/configuration/logging.properties
    jboss.modules.system.pkgs=org.jboss.byteman,mussarela.sun.maionese.batatinha
    org.apache.xml.security.ignoreLineBreaks=true
    sun.java.launcher=SUN_STANDARD
    sun.management.compiler=HotSpot 64-Bit Tiered Compilers
    sun.nio.ch.bugLevel=
    org.jboss.security.web.audit=headers,cookies,parameter
    os.name=Linux
    sun.boot.class.path=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/resources.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/rt.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/sunrsasign.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/jsse.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/jce.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/charsets.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/lib/jfr.jar\:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre/classes
    
  6. Generating a GC histogram, use GC.class_histogram:

        $ jcmd 26670 GC.class_histogram
        26670:
    
         num     #instances         #bytes  class name
        ----------------------------------------------
           1:        101528       13179464  [C
           2:        197580        6322560  java.util.HashMap$Node
           3:        100366        2408784  java.lang.String
           4:         10770        2130368  [Ljava.util.HashMap$Node;
           5:         17441        1967584  java.lang.Class
           6:         30017        1513200  [Ljava.lang.Object;
           7:          6354        1113232  [B
           8:          7572         726912  java.util.jar.JarFile$JarFileEntry
           9:         12880         618240  java.util.HashMap
          10:         16885         405240  java.util.ArrayList
          11:          6641         389504  [I
          12:          9354         374160  java.util.LinkedHashMap$Entry
          13:         10629         340128  java.util.concurrent.ConcurrentHashMap$Node
          14:         12039         288936  java.util.Collections$UnmodifiableRandomAccess
    

    This provides heap usage by listing all, including the number of instances and the respective heap usage. The output is ordered by the size in the heap and one can dump in a file to get the complete list. For class_stats, though, it is necessary to enable UnlockDiagnosticVMOptions
    The large memory users shown in the histogram are just generic objects like byte arrays, Strings, and HashMap entries, so this isn't sufficient. In most cases, a full heap dump that includes Dominator Tree is often the most useful in tracking down high-memory users, except for specific use cases.

For Native Memory leaks

Combined with -XX:NativeMemoryTracking with detail or summary it will provide much information about memory, as detailed in the solution JVM Native Memory Tracking.

Generating Thread dumps

It can also be used to generate thread dumps, as detailed in Generate a Java thread dump on Microsoft Windows.

jcmd $PID  Thread.print

Generating Heap dumps

And to generate heap dump as well. And must be used with the same java user.

jcmd $PID GC.heap_dump heap.hprof

Other options:

OpenJDK 11/17 has several other options, Content from bugs.openjdk.org is not included.Compiler, ManagementAgent, cds, Content from bugs.openjdk.org is not included.System.trim_native_heap:

sh-4.4$ jcmd $PID help
152:
The following commands are available:
Compiler.CodeHeap_Analytics
Compiler.codecache
Compiler.codelist
Compiler.directives_add
Compiler.directives_clear
Compiler.directives_print
Compiler.directives_remove
Compiler.perfmap
Compiler.queue
GC.class_histogram
GC.finalizer_info
GC.heap_dump
GC.heap_info
GC.run
GC.run_finalization
JFR.check
JFR.configure
JFR.dump
JFR.start
JFR.stop
JVMTI.agent_load
JVMTI.data_dump
ManagementAgent.start
ManagementAgent.start_local
ManagementAgent.status
ManagementAgent.stop
System.trim_native_heap
Thread.print
VM.cds
VM.class_hierarchy
VM.classloader_stats
VM.classloaders
VM.command_line
VM.dynlibs
VM.events
VM.flags
VM.info
VM.log
VM.metaspace
VM.native_memory
VM.print_touched_methods
VM.set_flag
VM.stringtable
VM.symboltable
VM.system_properties
VM.systemdictionary
VM.uptime
VM.version
help
Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.