Java JVM memory allocation pattern explained

How does the Java JVM allocate memory?

Java objects reside in an area of memory called the heap. Whenever we create an object, it is created in the heap space. Any object created in the heap space has global access and can be referenced from anywhere on the application. When the heap becomes full, garbage is collected, and during the garbage collection process, objects that are no longer in use are cleared, thus making space for new objects.

Automatic garbage collection is the process of looking at heap memory, identifying which objects are in use and which are not, and deleting the unused objects. An in use object, or a referenced object, means that some part of the application still maintains a pointer to that object. An unused object, or unreferenced object, is no longer referenced by any part of the application. So the memory used by an unreferenced object can be reclaimed.

The heap is separated into different areas often referred to as "generations". The generations are: Young Generation (divided further as eden, S0 and S1), Old or Tenured Generation, and Permanent generation.

The Young Generation is where all new objects are allocated and aged. When the young generation fills up, this causes a minor garbage collection. Minor collections can be optimized assuming a high object mortality rate. A young generation full of dead objects is collected very quickly. Some surviving objects are aged and eventually move to the old generation.

Stop the World Event - All minor garbage collections are "Stop the World" events. This means that all application threads are stopped until the operation completes. Minor garbage collections are always Stop the World events.

The Old Generation is used to store long surviving objects. Typically, a threshold is set for young generation objects and when that age is met, the object gets moved to the old generation. Eventually the old generation needs to be collected. This event is called a major garbage collection.

Major garbage collection are also Stop the World events. Often a major collection is much slower because it involves all live objects. So for responsive applications, major garbage collections should be minimized.

The Permanent generation contains metadata required by the JVM to describe the classes and methods used in the application. The permanent generation is populated by the JVM at runtime based on classes in use by the application. In addition, Java SE library classes and methods may be stored here.

Classes may get collected (unloaded) if the JVM finds they are no longer needed and space may be needed for other classes. The permanent generation is included in a full garbage collection.

This is a summary of how the JVM allocates memory:

1. First, any new objects are allocated to the eden space. Both survivor spaces start out empty.
2. When the eden space fills up, a minor garbage collection is triggered.
3. Referenced objects are moved to the first survivor space. Unreferenced objects are deleted when the eden space is cleared.
4. At the next minor garbage collection, the same thing happens for the eden space. Unreferenced objects are deleted and referenced objects are moved to a survivor space. However, in this case, they are moved to the second survivor space (S1). In addition, objects from the last minor GC on the first survivor space (S0) have their age incremented and get moved to S1. Once all surviving objects have been moved to S1, both S0 and eden are cleared. Notice we now have differently aged object in the survivor space.
5. At the next minor GC, the same process repeats. However this time the survivor spaces switch. Referenced objects are moved to S0. Surviving objects are aged. Eden and S1 are cleared.
6. After a minor GC, when aged objects reach a certain age threshold, they are promoted from young generation to old generation.
7. As minor GCs continue to occure objects will continue to be promoted to the old generation space.
8. The steps above cover the entire process with the young generation. Eventually, a major GC will be performed on the old generation which cleans up and compacts that space.

Important considerations

1. The JVM heap usage usually follows a "sawtooth" pattern. The memory in use can grow high and then be recollected, which is the expected behavior and does not indicate a memory leak.
2. Normally, when there is a Out of Memory condition, the CPU usage increases due to full garbage collection events, and this may turn the application unresponsive.
3. When deploying applications to CloudHub, it is important to take into account the worker size (vCores allocated for the worker). Especially considering that workers that use a fraction of a vCore have limited memory, CPU and IO resources (see the CloudHub documentation for details https://docs.mulesoft.com/runtime-manager/cloudhub-architecture#cloudhub-workers). Make sure to stress test applications before deploying to a production environment.

Useful links

1. Official Oracle Java Garbage Collection Basics guide: http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html
2. How to gather a heap dump on JVM Out of Memory error for a standalone mule runtime: https://help.salesforce.com/s/articleView?id=How-to-obtain-as-much-information-as-possible-from-unresponsive-Mule-server&type=1&language=en_US
3. VisualVM (open source UI tool that integrates JDK command line tools and lightweight profiling capabilities for Java JVMs): https://visualvm.github.io/. A must have tool for application performance analysis, diagnostics and debugging.