JVM Native Memory Tracking
Environment
- OpenJDK 8
- OpenJDK 11
Issue
- How to use
NativeMemoryTrackingflag? - What options there are for
NativeMemoryTracking?
Resolution
Disclaimer: Links contained herein to an external website(s) are provided for convenience only. Red Hat has not reviewed the links and is not responsible for the content or its availability. The inclusion of any link to an external website does not imply endorsement by Red Hat of the website or their entities, products or services. You agree that Red Hat is not responsible or liable for any loss or expenses that may result due to your use of (or reliance on) the external site or content.
The tool, Native Memory Tracking (NMT) is used to tracks internal memory usage. It has three options: off, summary, and detail:
-XX:NativeMemoryTracking=[off | summary | detail]
Using jcmd tool for tracking the values
After enabling NativeMemoryTracking=detail combined with jcmd, it can be used to show the allocations including the heap, class metadata, application code, the code generated by JIT, internal data structures, and GC:
JDK8_HOME/bin/jcmd 14755 VM.native_memory
14755:
Native Memory Tracking:
Total: reserved=2927744KB, committed=1664804KB <---------Total reserved and commited data
- Java Heap (reserved=1335296KB, committed=1335296KB) <---------Heap
(mmap: reserved=1335296KB, committed=1335296KB)
- Class (reserved=1154720KB, committed=122400KB) <--------------Metaspace
(classes #19813)
(malloc=11936KB #26862)
(mmap: reserved=1142784KB, committed=110464KB) <-----------committed
- Thread (reserved=82535KB, committed=82535KB) <----------------Threads
(thread #81)<---------81 threads
(stack: reserved=82180KB, committed=82180KB) <---------------82mb
(malloc=261KB #410)
(arena=94KB #156)
- Code (reserved=253164KB, committed=22544KB) <----------------Code Cached
(malloc=3564KB #6105)
(mmap: reserved=249600KB, committed=18980KB)
- GC (reserved=59188KB, committed=59188KB)<-------------------GC memory usage
(malloc=10396KB #526)
(mmap: reserved=48792KB, committed=48792KB)
- Compiler (reserved=259KB, committed=259KB) <--------------------Compiler
(malloc=128KB #692)
(arena=131KB #7)
- Internal (reserved=14574KB, committed=14574KB) <------------------Internal
(malloc=14542KB #25518)
(mmap: reserved=32KB, committed=32KB)
- Symbol (reserved=23051KB, committed=23051KB) <-----------------Symbol
(malloc=20870KB #226280)
(arena=2182KB #1)
- Native Memory Tracking (reserved=4758KB, committed=4758KB)
(malloc=229KB #3243)
(tracking overhead=4529KB)
- Arena Chunk (reserved=198KB, committed=198KB)
(malloc=198KB)
Interpreting the fields on NMT
The Native Memory Tracker has the following categories:
| Field | Meaning |
|---|---|
| Java Heap | Heap space |
| Class | native memory allocated for the table of |
| Thread | native memory allocated for the application threads |
| Code | native memory allocated for the code is being executed |
| GC | native memory used by the Garbage Collector |
| Compiler | native memory allocated for the compiler operations |
| Internal | internal allocations |
| Symbol | native memory allocated for the table of symbols |
| Native Memory Tracking | native memory used for the NMT processes |
| Arena Chunk | allocations for the arena-managed chunk |
| Others | Includes off-heap memory allocated with Unsafe.allocateMemory and Direct ByteBuffers. |
| Module | Memory used by modules(module JDK 9+) |
| Unknown | unspecified allocations |
For more information, see Content from docs.oracle.com is not included.Oracle Java8 - NMT Memory Categories.
JDK 11 new option for Metaspace
For JDK 11, there is also the option for VM.metaspace, which is not part of the heap:
JDK11_home/bin/jcmd 27689 VM.metaspace
27689:
Total Usage ( 1475 loaders):
Non-Class: 4306 chunks, 99.12 MB capacity, 92.54 MB ( 93%) used, 6.31 MB ( 6%) free, 4.66 KB ( <1%) waste, 269.12 KB ( <1%) overhead, deallocated: 42488 blocks with 6.34 MB
Class: 2387 chunks, 16.99 MB capacity, 13.64 MB ( 80%) used, 3.20 MB ( 19%) free, 328 bytes ( <1%) waste, 149.19 KB ( <1%) overhead, deallocated: 2151 blocks with 1.21 MB
Both: 6693 chunks, 116.11 MB capacity, 106.18 MB ( 91%) used, 9.51 MB ( 8%) free, 4.98 KB ( <1%) waste, 418.31 KB ( <1%) overhead, deallocated: 44639 blocks with 7.55 MB
Virtual space:
Non-class space: 100.00 MB reserved, 99.50 MB (>99%) committed
Class space: 248.00 MB reserved, 17.15 MB ( 7%) committed
Both: 348.00 MB reserved, 116.65 MB ( 34%) committed
Chunk freelists:
Non-Class:
specialized chunks: 11, capacity 11.00 KB
small chunks: 56, capacity 224.00 KB
medium chunks: (none)
humongous chunks: (none)
Total: 67, capacity=235.00 KB
Class:
specialized chunks: 3, capacity 3.00 KB
small chunks: 20, capacity 40.00 KB
medium chunks: (none)
humongous chunks: (none)
Total: 23, capacity=43.00 KB
Waste (percentages refer to total committed size 116.65 MB):
Committed unused: 276.00 KB ( <1%)
Waste in chunks in use: 4.98 KB ( <1%)
Free in chunks in use: 9.51 MB ( 8%)
Overhead in chunks in use: 418.31 KB ( <1%)
In free chunks: 278.00 KB ( <1%)
Deallocated from chunks in use: 7.55 MB ( 6%) (44639 blocks)
-total-: 18.01 MB ( 15%)
MaxMetaspaceSize: 256.00 MB
InitialBootClassLoaderMetaspaceSize: 4.00 MB
UseCompressedClassPointers: true
CompressedClassSpaceSize: 248.00 MB
Baseline comparison
Given a process with the flags -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=detail on the process, a baseline can be taken via baseline in VM.native_memory:
jcmd PID VM.native_memory baseline
jcmd PID VM.native_memory detail.diff >> jcmd_native_diff.out
The output will show the increase or decrease:
Total: reserved=2028691KB +4144KB, committed=324019KB +4212KB <----
- Java Heap (reserved=524288KB, committed=163840KB)
(mmap: reserved=524288KB, committed=163840KB)
- Class (reserved=1103702KB, committed=61398KB)
(classes #10621)
( instance classes #10065, array classes #556)
(malloc=1878KB #26250 +15)
(mmap: reserved=1101824KB, committed=59520KB)
( Metadata: )
( reserved=53248KB, committed=51712KB)
( used=50400KB +22KB)
( free=1312KB -22KB)
Trimming
The JVM flag TrimNativeHeapInterval, which was experimental, provides the ability of the JVM to trim off-heap memory such as glibc allocation by itself. This means to say: the JVM will return back to the kernel the memory that was used and not anymore.
This flag sets an interval, in ms, at which the JVM will trim the native heap. The lower the values, the faster the thread will try to collect the memory and where the value zero indicates disabled.
Root Cause
For Openshift, see Java's memory consumption inside a Openshift 4 container.
Diagnostic Steps
Use the solution Example of native allocation tracking from VM.info and Native tracking in OpenJDK as example for analysis.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.