18286, "ronawho", "Using a fixed heap can lead to fragmentation", "2021-08-24T19:27:22Z"
Today we have 3 different memory allocation modes:
- simple: just use jemalloc as a replacement for the system allocator
- dynamic-heap: large arrays are allocated through the comm layer. Anything else is done with dynamic extensions to a heap. there are jemalloc chunk hooks that ask the comm layer for more memory as needed for the dynamic extensions.
- fixed-heap: we get a blob of registered memory from the comm layer and we provide jemalloc with chunk hooks to satisfy allocations from that blob
For the simple case fragmentation is handled entirely by the memory layer and jemalloc is typically really good at fragmentation avoidance. For the dynamic heap large arrays are separately allocated and free'd back to the OS so fragmentation isn't much of an issue here either. Non-large array allocations suffer slightly from fragmentation because we don't provide hooks for jemalloc to return memory or to split/merge existing chunks but this isn't bad.
The fixed heap can lead to severe fragmentation. What we provide is effectively a sbrk like interface where we can only bump a pointer into this fixed heap to satisfy new allocations but we're not providing jemalloc deallocation or merge/split hooks. This can result in cases where if you allocate a 100G array and then a 101G array that first 100G region can't be reused. Similarly if you allocate a 100G array and then 2 50G arrays we can't split that first 100G so 50G is wasted. Additionally since large allocations and small allocations are intertwined we can get additional fragmentation.
We should look at providing merge/split hooks so that jemalloc can better manage existing chunks. It may also be worthwhile to satisfy small allocations starting at the top of the heap and larger ones from the bottom to limit fragmentation between long-lived small allocations (e.g. task stacks) and larger arrays. We may also be able to provide deallocation hooks, though I'm not sure how necessary that is if we have the merge/split functionality.