JVM Native Memory Tracking

Solution Verified - Updated

Environment

  • OpenJDK 8
  • OpenJDK 11

Issue

  • How to use NativeMemoryTracking flag?
  • What options there are for NativeMemoryTracking?

Resolution


Disclaimer: Links contained herein to an external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.


The tool, Native Memory Tracking (NMT) is used to tracks internal memory usage. It has three options: off, summary, and detail:

-XX:NativeMemoryTracking=[off | summary | detail]

Using jcmd tool for tracking the values

After enabling NativeMemoryTracking=detail combined with jcmd, it can be used to show the allocations including the heap, class metadata, application code, the code generated by JIT, internal data structures, and GC:

JDK8_HOME/bin/jcmd 14755 VM.native_memory
14755:

Native Memory Tracking:

Total: reserved=2927744KB, committed=1664804KB <---------Total reserved and commited data
-                 Java Heap (reserved=1335296KB, committed=1335296KB) <---------Heap
                            (mmap: reserved=1335296KB, committed=1335296KB) 
 
-                     Class (reserved=1154720KB, committed=122400KB) <--------------Metaspace
                            (classes #19813)
                            (malloc=11936KB #26862) 
                            (mmap: reserved=1142784KB, committed=110464KB)  <-----------committed
 
-                    Thread (reserved=82535KB, committed=82535KB) <----------------Threads
                            (thread #81)<---------81 threads
                            (stack: reserved=82180KB, committed=82180KB) <---------------82mb 
                            (malloc=261KB #410) 
                            (arena=94KB #156)
 
-                      Code (reserved=253164KB, committed=22544KB) <----------------Code Cached
                            (malloc=3564KB #6105) 
                            (mmap: reserved=249600KB, committed=18980KB) 
 
-                        GC (reserved=59188KB, committed=59188KB)<-------------------GC memory usage
                            (malloc=10396KB #526) 
                            (mmap: reserved=48792KB, committed=48792KB) 
 
-                  Compiler (reserved=259KB, committed=259KB) <--------------------Compiler
                            (malloc=128KB #692) 
                            (arena=131KB #7)
 
-                  Internal (reserved=14574KB, committed=14574KB) <------------------Internal
                            (malloc=14542KB #25518) 
                            (mmap: reserved=32KB, committed=32KB) 
 
-                    Symbol (reserved=23051KB, committed=23051KB) <-----------------Symbol
                            (malloc=20870KB #226280) 
                            (arena=2182KB #1)
 
-    Native Memory Tracking (reserved=4758KB, committed=4758KB)
                            (malloc=229KB #3243) 
                            (tracking overhead=4529KB)
 
-               Arena Chunk (reserved=198KB, committed=198KB)
                            (malloc=198KB) 

Interpreting the fields on NMT

The Native Memory Tracker has the following categories:

FieldMeaning
Java HeapHeap space
Classnative memory allocated for the table of
Threadnative memory allocated for the application threads
Codenative memory allocated for the code is being executed
GCnative memory used by the Garbage Collector
Compilernative memory allocated for the compiler operations
Internalinternal allocations
Symbolnative memory allocated for the table of symbols
Native Memory Trackingnative memory used for the NMT processes
Arena Chunkallocations for the arena-managed chunk
OthersIncludes off-heap memory allocated with Unsafe.allocateMemory and Direct ByteBuffers.
ModuleMemory used by modules(module JDK 9+)
Unknownunspecified allocations

For more information, see Content from docs.oracle.com is not included.Oracle Java8 - NMT Memory Categories.

JDK 11 new option for Metaspace

For JDK 11, there is also the option for VM.metaspace, which is not part of the heap:

JDK11_home/bin/jcmd 27689 VM.metaspace
27689:

Total Usage ( 1475 loaders):

  Non-Class: 4306 chunks,     99.12 MB capacity,    92.54 MB ( 93%) used,     6.31 MB (  6%) free,     4.66 KB ( <1%) waste,   269.12 KB ( <1%) overhead, deallocated: 42488 blocks with 6.34 MB
      Class: 2387 chunks,     16.99 MB capacity,    13.64 MB ( 80%) used,     3.20 MB ( 19%) free,   328 bytes ( <1%) waste,   149.19 KB ( <1%) overhead, deallocated: 2151 blocks with 1.21 MB
       Both: 6693 chunks,    116.11 MB capacity,   106.18 MB ( 91%) used,     9.51 MB (  8%) free,     4.98 KB ( <1%) waste,   418.31 KB ( <1%) overhead, deallocated: 44639 blocks with 7.55 MB

Virtual space:
  Non-class space:      100.00 MB reserved,      99.50 MB (>99%) committed 
      Class space:      248.00 MB reserved,      17.15 MB (  7%) committed 
             Both:      348.00 MB reserved,     116.65 MB ( 34%) committed 



Chunk freelists:
   Non-Class:

 specialized chunks:   11, capacity 11.00 KB
       small chunks:   56, capacity 224.00 KB
      medium chunks: (none)
   humongous chunks: (none)
              Total:   67, capacity=235.00 KB
       Class:

 specialized chunks:    3, capacity 3.00 KB
       small chunks:   20, capacity 40.00 KB
      medium chunks: (none)
   humongous chunks: (none)
              Total:   23, capacity=43.00 KB

Waste (percentages refer to total committed size 116.65 MB):
              Committed unused:    276.00 KB ( <1%)
        Waste in chunks in use:      4.98 KB ( <1%)
         Free in chunks in use:      9.51 MB (  8%)
     Overhead in chunks in use:    418.31 KB ( <1%)
                In free chunks:    278.00 KB ( <1%)
Deallocated from chunks in use:      7.55 MB (  6%) (44639 blocks)
                       -total-:     18.01 MB ( 15%)


MaxMetaspaceSize: 256.00 MB
InitialBootClassLoaderMetaspaceSize: 4.00 MB
UseCompressedClassPointers: true
CompressedClassSpaceSize: 248.00 MB

Baseline comparison

Given a process with the flags -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=detail on the process, a baseline can be taken via baseline in VM.native_memory:

jcmd PID VM.native_memory baseline
jcmd PID VM.native_memory detail.diff >> jcmd_native_diff.out

The output will show the increase or decrease:

Total: reserved=2028691KB +4144KB, committed=324019KB +4212KB <----

-                 Java Heap (reserved=524288KB, committed=163840KB)
                            (mmap: reserved=524288KB, committed=163840KB)
 
-                     Class (reserved=1103702KB, committed=61398KB)
                            (classes #10621)
                            (  instance classes #10065, array classes #556)
                            (malloc=1878KB #26250 +15)
                            (mmap: reserved=1101824KB, committed=59520KB)
                            (  Metadata:   )
                            (    reserved=53248KB, committed=51712KB)
                            (    used=50400KB +22KB)
                            (    free=1312KB -22KB)

Trimming

The JVM flag TrimNativeHeapInterval, which was experimental, provides the ability of the JVM to trim off-heap memory such as glibc allocation by itself. This means to say: the JVM will return back to the kernel the memory that was used and not anymore.
This flag sets an interval, in ms, at which the JVM will trim the native heap. The lower the values, the faster the thread will try to collect the memory and where the value zero indicates disabled.

Root Cause

For Openshift, see Java's memory consumption inside a Openshift 4 container.

Diagnostic Steps

Use the solution Example of native allocation tracking from VM.info and Native tracking in OpenJDK as example for analysis.

Category

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.