Harvest Java Heap Dumps in constrained environments

Fri, 20 Jan 2023 20:40:30 +0100

Creating and analyzing Heap Dumps is a standard for analyzing memory issues with JVM based programs. To do this, you have to:

create the Heap Dump (duh!)
move the data to your computer
actually analyze the data

Each of these steps has it's own challenges though the first two seem trivial at first sight.

Creating the Heap Dump

The main challenge in Heap Dump creation I faced in today's production environments is that only essential software is installed in order to keep containers and the potential attack surface small. So likely there is only a JVM which is enough to run the program but not enough to get a Heap Dump. Through the recommendation of a colleague I became aware of jattach. It describes itself as:

Why does MaxRAMPercentage stop working at heaps of ~ 30 GB?

Sat, 22 Jan 2022 18:30:50 +0100

The other day at work a Clojure program on a machine with 100 GB ran into OutOfMemory errors. A quick look at the metrics showed that the heap was only at about 30 GB though. Checking the JVM flags I found a configuration that is commonplace:

-XX:MaxRAMPercentage=80

So one would assume that the JVM could have sized the heap much larger than it actually did. Thanks to docker this behavior can be observed quite easily by doing a few tests like this one:

Jvm on Personal weblog

Harvest Java Heap Dumps in constrained environments

Creating the Heap Dump

Why does MaxRAMPercentage stop working at heaps of ~ 30 GB?