What’s the difference between On heap and Off Heap in Java - JVM Memory

Question

I would like an explanation determining the characteristics of On heap and Off heap Memory in Java.

We know that Thread Stacks, the code of our application and the Buffers NIO are all allocated Off heap. But after all, what does this term mean?

What are the semantics of memory allocation On Heap vs Off Heap?

Answer 1

Hello @Filipegonzagamiranda, follow my view on your question;

I would like an explanation determining the characteristics of On heap and Off heap Memory in Java.

On-Heap Java Memory is the memory managed by Java Virtual Machine (JVM), java heap is established at the startup of the virtual machine process, and its size can be specified at that time;

java -Xmx2048m -Xms256m (... meus argumentos jvm ...)

Therefore java heap is a memory space made available to the JVM instance for use during the execution of the program (bytecode), the heap in question is shared with all Threads belonging to JVM instance (process).

But what effectively "stays" in the Java Heap ?

• All class instances created by the application.

Being direct then we have Java Heap as a storage point for all memory allocations made by the JVM instance during its execution.

What are the most striking characteristics of this "storage point"?

• Automated management "the cleanliness" of memory. (or Garbage Collection)
• No need to allocate memory continues for the elements (objects) inside the Heap.
• Heap space allocation (effective occupation) occurs dynamically (for greater or lesser), respecting the limits specified in JVM startup and OS/Architecture addressing.

We know that Thread Stacks, our application code and the Buffers NIO are all allocated Off heap. But after all, what does this mean term?

To understand more deeply the concept of "On Heap" and "Off Heap" let’s view the execution context of the JVM instance:

As you observed, a Thread Stack (or better Java Virtual Machine Stack) would be "Off Heap" by the fact of its execution area within a Java Thread and not occur within the Heap.

Now as to Buffers NIO the subject begins to narrow, clearly there is no "Runtime data area" specifies (in the JVM architecture) to allocate memory outside the jvm heap natively. Let’s say I wanted to cache an extensive set (Set) of information, normally we would keep a static instance in the heap which all Threads could reference it "On heap".

What about "new I/O Apis" or Bytebuffer and the "Off Heap" ?
To New I/O Apis delivery of a potential resource to an implementation "Off Heap" Memory Mapped File, that’s the direct buffers.
His key points are:

• The contents of a direct buffer may reside in native memory, i.e., outside the heap managed by jvm.
• Each JVM must implement natively new operations:
  • Newdirectbytebuffer
  • Getdirectbufferaddress
  • Getdirectbuffercapacity
• Direct buffers operations are defined by a Bytebuffer.

Why not "On Heap"?

If an application requires a bulky set of information to the point that it is not practical, or feasible to manage it in heap, by the excess of breaks caused by Garbage Colletor (or by conflicting GC strategies in areas other than Heap) that can leave us in the face of java heap memory Leak.

Another point would be regarding possible problems to a non predictable (or incompatible) memory addressing volume (remember for object allocation in heap is not continuous, but for JVM yes, heap range-size must be specified in JVM process creation).

Just as if we wanted to share this same heap between instances of different Vms.

In such cases strategy would be the memory allocation "Off Heap".

The definitive feature of allocation "Off Heap" is that she doesn’t share the Heap of JVM native architecture and all its features and management by it. (Garbage Collection for example)

What are the memory allocation semantics On Heap vs Off Heap?

- semantics of allocation On Heap:

public class OffHeap {
    public static void main(String[] args) {
        // TestObject é criado no jvm heap usando o operador new
        // t variavel local no stack recebe referencia de TestObject
        TestObject t = new TestObject();
    }
}

- semantics of allocation Off Heap:
I’ll use an example of Peter Lawrey in stack overflow Java - Swapping Page. In this code is created a custom Hashmap supporting being paged in memory files "Off Heap".

class LongIntParallelHashMultimap {
    private static final int NULL = 0;
    private final FileChannel channel1, channel2;
    private final LongBuffer keys;
    private final IntBuffer values;
    private final int capacity;
    private int size;

    public LongIntParallelHashMultimap(int capacity, String basename) throws IOException {
        assert (capacity & (capacity - 1)) == 0 : "Capacity " + capacity + " must be a power of 2";
        this.capacity = capacity;
        channel1 = new RandomAccessFile(basename + ".keys", "rw").getChannel();
        keys = channel1.map(FileChannel.MapMode.READ_WRITE, 0, capacity * 8).order(ByteOrder.nativeOrder()).asLongBuffer();
        // load keys into memory
        for(int i=0;i<capacity;i+=512) keys.get(i);

        channel2 = new RandomAccessFile(basename + ".values", "rw").getChannel();
        values = channel2.map(FileChannel.MapMode.READ_WRITE, 0, capacity * 4).order(ByteOrder.nativeOrder()).asIntBuffer();
        for(int i=0;i<capacity;i+=1024) values.get(i);
    }

    public void put(long key, int value) {
        long key1 = key + 1;
        int index = indexFor(key1);
        while (keys.get(index) != NULL) {
            index = successor(index);
        }
        values.put(index, value);
        keys.put(index, key1);
        ++size;
    }

    /**
     * Uses a pre-allocated array and return the count of matches.
     */
    public int get(long key, int[] hits) {
        long key1 = key + 1;
        int index = indexFor(key1);
        int hitIndex = 0;

        while (keys.get(index) != NULL) {
            if (keys.get(index) == key1) {
                hits[hitIndex] = values.get(index);
                ++hitIndex;
            }
            index = successor(index);
        }

        return hitIndex;
    }

    private int indexFor(long key) {
        return Math.abs((int) ((key * 5700357409661598721L) & (capacity - 1)));
    }

    private int successor(int index) {
        return (index + 1) & (capacity - 1);
    }

    public int size() {
        return size;
    }

    public void close() {
        try {
            channel1.close();
            channel2.close();
        } catch (IOException ignored) {
        }
        try {
            ((DirectBuffer) keys).cleaner().clean();
            ((DirectBuffer) values).cleaner().clean();
        } catch (Throwable notSupportedOnThisPlatform) {
        }
    }
}

Context of implementation:

long heap = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
RandomAccessFile raf = new RandomAccessFile("array.dat", "rw");
IntBuffer map = raf.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, 1 << 30).order(ByteOrder.nativeOrder()).asIntBuffer();
for (int i = 0; i < map.capacity(); i++)
    map.put(i, i);
long heap2 = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
System.out.printf("Wrote %,d int values, heap used %,d bytes approx%n", map.capacity(), heap2 - heap);

Upshot:

Wrote 268,435,456 int values, heap used 0 approx

Answer 2

JVM has a heap / free store of defined size, and this size can be altered. In C++, to allocate memory in the heap, use the operator new (If you do not use the new operator, the memory is allocated to the stack of size defined at the time of execution of the program). In Java, all memory allocation is done in heap by default (it is not possible to allocate in stack), and GC / Garbage Collector is responsible for "cleaning" and checking it (So Java kept the convention of using the new operator, even though it is unnecessary / useless in Java).

The difference here is that when using C/C++, the allocation is done in the OS heap, and in Java, the allocation is done in the JVM heap. When this limit size is exceeded in a very large application, it is possible to allocate memory in the OS heap through serialization, and so it is called off-heap.

On-heap: Heap of the JVM. Off-heap: Heap of OS.

EDIT:

In Stack Overflow they often comment on why to negative an answer. Why, and how, my answer does not answer the question?