Loading

How to monitor JVM native memory usage by enabling native memory tracking

Publiceringsdatum: Oct 15, 2025
Uppgift

GOAL

The HotSpot (r) JVM uses two well defined memory spaces: the heap space and the native space. The former is the space where application allocation takes place, and is comprised mainly by the heap generations ( E, S0, S1, tenured space ) and the permanent generation or PermGen (only for Java 7 and earlier).

Heap space is a well delimited memory arena which is reserved during JVM boot-up, and grows until a maximum reserved size (-Xmx) is reached.

Native space is not confined to any particular space, and consists of memory that resides out of the JVM memory space allocated by the O/S, which is allocated through native invocations at the operating system level. In theory, a JVM can allocate as much memory as physical memory is available on the machine, or until resource limits are reached by O/S quota control.

This is the space where the JVM allocates internal objects of the following type:
  • Loaded libraries
  • NIO buffers ( i.e., HeapByteBuffers )
  • Code cache ( JIT compiler optimizations), String Cache Pool
  • Thread Stacks
  • Class Metadata  (in metaspace, since Java 8)
 
Due to the fact that native heap is not part of a heap dump, the native memory cannot be directly analyzed using a Java memory profiler, so basically native memory usage remains as a mysterious black box whose content cannot be completely discovered. 

At this point is where the native memory tracking feature comes into help, which is included on Java since release 7u40 (1.7.0_40) onwards.

Native memory tracking allows a JVM to profile and track native space usage inside a JVM, providing statistics on the different kind of objects being allocated, providing a summarized report.
Steg

To enable native memory tracking on java 7 and 8, just add one of the following options on your $MULE_HOME/conf/wrapper.conf mule wrapper configuration file:

wrapper.java.additional.<n>=-XX:NativeMemoryTracking=detail    # for a detailed report

or

wrapper.java.additional.<n>=-XX:NativeMemoryTracking=summary   # for a summarized report

Note: as usual when adding Java properties to wrapper.conf, you need to replace the <n> placeholder for a suitable available number. See KB article How can I set Mule and Java system properties at startup? for more details.

The Mule instance will need to be restarted for the change to take effect.

After the restart, you can execute a native memory tracking report, by issuing the following command:
 

$ $JAVA_HOME/bin/jcmd <pid of mule JVM> VM.native_memory detail

Note: the examples assume the Linux environment variable JAVA_HOME points to the Java JDK installation directory. You can replace it directly by the complete path the commands. Same applies for Windows, except that the convention for environment variables uses characters '%' enclosing the variable name, instead of a '$' at the beginning. Eg: %JAVA_HOME%.

If you want to make comparison on native memory usage to a certain point in time, you can use baselines, as shown below:

$ $JAVA_HOME/bin/jcmd <pid of mule JVM> VM.native_memory baseline


Once you have defined the baseline, you can query the actual memory usage compared with the baseline, by issuing the following command:

$ $JAVA_HOME/bin/jcmd <pid of mule JVM> VM.native_memory detail.diff


This will give you relative values of memory, with respect to the baseline, which is very useful when trying to identify how fast native memory grows over time.

Please refer to the following Oracle's official document for additional information on tracking native memory:
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html

Enabling native memory tracking at JVM shutdown

You can enable a native memory tracking report to be generated automatically when the JVM shuts down.
To accomplish that, you should add the following option on your $MULE_HOME/conf/wrapper.conf file:

wrapper.java.additional.<n>=-XX:+UnlockDiagnosticVMOptions
wrapper.java.additional.<n>=-XX:+PrintNMTStatistics

Depending on the configure value ("detail" or "summary") of the "-XX:NativeMemoryTracking" parameter, a corresponding native memory tracking report will be generated and appended to the end of the $MULE_HOME/logs/mule_ee.log file.

Here is an example of Native Memory Tracking summary:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32744 bytes. Error detail: ChunkPool::allocate
# An error report file with more information is saved as:
# /apps/mule/mycoprd/mule-clst1/lib/boot/tanuki/exec/hs_err_pid11075992.log

Native Memory Tracking:

Total: reserved=30777863579, committed=25270611355
       malloc: 17004424603 #2969994
       mmap:   reserved=13773438976, committed=8266186752

-                 Java Heap (reserved=10737418240, committed=6333399040)
                            (mmap: reserved=10737418240, committed=6333399040) 
 
-                     Class (reserved=1098878319, committed=212569455)
                            (classes #232991)
                            (  instance classes #223785, array classes #9206)
                            (malloc=25136495 #562493) (peak=25177183 #563537) 
                            (mmap: reserved=1073741824, committed=187432960) 
                            (  Metadata:   )
                            (    reserved=1275068416, committed=1224146944)
                            (    used=1220106136)
                            (    waste=4040808 =0.33%)
                            (  Class space:)
                            (    reserved=1073741824, committed=187432960)
                            (    used=183649552)
                            (    waste=3783408 =2.02%)
 
-                    Thread (reserved=16418916256, committed=16418916256)
                            (thread #7509)
                            (Stack: 16246324520)
                            (malloc=11797688 #45168) (peak=12181968 #52945) 
                            (arena=160794048 #15014) (peak=195979288 #14910)
 
-                      Code (reserved=267287252, committed=264534740)
                            (malloc=13597396 #82198) (peak=14139876 #88778) 
                            (mmap: reserved=253689856, committed=250937344) 
 
-                        GC (reserved=542689859, committed=379505219)
                            (malloc=109300291 #130985) (peak=181439023 #91178) 
                            (mmap: reserved=433389568, committed=270204928) 
 
-                  Compiler (reserved=11061112, committed=11061112)
                            (malloc=10892552 #17269) (peak=11064728 #24439) 
                            (arena=168560 #5) (peak=127306512 #94)
 
-                  Internal (reserved=21666759, committed=21601223)
                            (malloc=21535687 #311294) (peak=22299131 #332099) 
                            (mmap: reserved=131072, committed=65536) 
 
-                     Other (reserved=265100712, committed=265100712)
                            (malloc=265100712 #7916) (peak=383482360 #6971) 
 
-                    Symbol (reserved=76251000, committed=76251000)
                            (malloc=72315008 #1697914) (peak=72315024 #1697915) 
                            (arena=3935992 #1) (at peak)
 
-    Native Memory Tracking (reserved=48862192, committed=48862192)
                            (malloc=1342288 #23950) (at peak) 
                            (tracking overhead=47519904)
 
-               Arena Chunk (reserved=373344, committed=373344)
                            (malloc=373344 #19784) (peak=204820136 #20935) 
 
-                   Tracing (reserved=1, committed=1)
                            (malloc=1 #1) (at peak) 
 
-                 Arguments (reserved=81, committed=81)
                            (malloc=81 #4) (peak=9321 #5) 
 
-                    Module (reserved=5631248, committed=5631248)
                            (malloc=5631248 #34576) (peak=5811760 #35090) 
 
-           Synchronization (reserved=1545844, committed=1545844)
                            (malloc=1545844 #12146) (peak=1552372 #12197) 
 
-            Serviceability (reserved=1786400, committed=1786400)
                            (malloc=1786400 #13432) (peak=2183616 #16411) 
 
-                 Metaspace (reserved=1280239456, committed=1229317984)
                            (malloc=5171040 #2886) (peak=5226976 #3070) 
                            (mmap: reserved=1275068416, committed=1224146944) 
 
-      String Deduplication (reserved=608, committed=608)
                            (malloc=608 #8) (at peak) 
 
-           Object Monitors (reserved=154896, committed=154896)
                            (malloc=154896 #461) (peak=830592 #2472)


 
Knowledge-artikelnummer
001114773
 
Laddar
Salesforce Help | Article