malloc internals — Why free() Doesn’t Return Memory to the System

Why Your Process’s RSS Grows and Doesn’t Shrink

A program allocates memory, processes data and calls free(), yet RSS in htop stays high. That symptom alone does not establish a leak. The allocator may retain freed blocks for reuse; distinguish them from memory the program has not freed.

In the vast majority of cases, free() does not return memory to the operating system. It returns it to the allocator — the intermediate layer between your code and the kernel that manages memory pools to avoid costly syscalls on every allocation. Understanding this layer is the difference between “we have a memory leak” and “that’s arena fragmentation, RSS won’t shrink, but it isn’t a leak.”

This article dissects the internals of malloc in glibc: arenas, bins, chunks, the brk vs mmap boundary, the mechanics of fragmentation, and — crucial for production decisions — when jemalloc or tcmalloc beats the default glibc allocator.

The Intermediate Layer: Why We Don’t Call the Kernel Directly

The kernel manages memory at page granularity (usually 4 KB). If every malloc(16) required a syscall to the kernel, the cost would be absurd — every syscall has overhead, and a typical program does millions of allocations. The allocator solves this by requesting memory from the kernel in bulk and dividing it into small pieces in userspace.

Two syscalls supply memory from the kernel:

# See both mechanisms in action — strace on a simple program
$ strace -e trace=brk,mmap ./program 2>&1 | head
brk(NULL)                = 0x55a3c2a00000        # current program break
brk(0x55a3c2a21000)      = 0x55a3c2a21000        # arena extension (132 KB)
mmap(NULL, 2101248, PROT_READ|PROT_WRITE, ...)   # large allocation (2 MB)
                         = 0x7f3a8c000000

M_MMAP_THRESHOLD influences the use of separate mappings for large allocations. The initial glibc threshold is normally 128 KiB and can change dynamically. Available free blocks and configuration also matter: this is not a fixed boundary for every malloc().

Anatomy of a Chunk — How malloc Remembers Sizes

When you call malloc(100), you get a pointer to 100 bytes — but the allocator reserved more. Every memory block is a chunk with a metadata header just before the returned pointer:

/* Simplified chunk structure in glibc (ptmalloc2) */
struct malloc_chunk {
    size_t      prev_size;  /* size of the previous chunk, IF free */
    size_t      size;       /* size of this chunk + 3 flag bits */

    /* The fields below are used ONLY when the chunk is free: */
    struct malloc_chunk *fd;  /* forward pointer in the free list */
    struct malloc_chunk *bk;  /* backward pointer in the free list */
};

/* The three lowest bits of 'size' are flags (alignment depends on the ABI): */
/* PREV_INUSE     (0x1) — whether the previous chunk is in use     */
/* IS_MMAPPED     (0x2) — whether the chunk comes from mmap         */
/* NON_MAIN_ARENA (0x4) — whether the chunk belongs to a thread arena */

While a chunk is allocated, part of the space later used for list pointers stores user data. A write after free() can therefore corrupt allocator metadata. Its consequences depend on the glibc version and protections; not every such write provides controlled access to an arbitrary address.

Bins — The Structure for Managing Free Chunks

Freed chunks aren’t immediately returned to the system. They go into bins — lists of free chunks grouped by size, so the next malloc() can quickly find a matching block. glibc maintains several bin categories with different characteristics:

Tcache (thread-local caching) is the most important optimization of recent years: each thread has its own pool of recently freed chunks, accessible without taking the arena lock. This dramatically speeds up multithreaded allocations, but also introduced new attack vectors (tcache poisoning).

Why free() Doesn’t Return Memory

The path taken by free() depends on the chunk and allocator state. A suitable block may enter tcache or a fast bin. Other blocks may be coalesced and placed in the unsorted bin before later classification; these are branches, not a sequence every block follows. Reclaimable memory may allow heap trimming, while an explicit malloc_trim() also tries to release complete free pages.

A live chunk after a free region can prevent brk from retreating. That is not the only way to reclaim RAM: glibc’s malloc_trim() can also release whole free pages inside arenas through madvise(). Reserved virtual addresses and resident physical memory are different quantities.

#include <stdlib.h>
#include <malloc.h>

int main(void) {
    /* Allocate 1000 blocks of 1 KB */
    void *blocks[1000];
    for (int i = 0; i < 1000; i++)
        blocks[i] = malloc(1024);

    /* Free all the EVEN ones — a fragmentation checkerboard */
    for (int i = 0; i < 1000; i += 2)
        free(blocks[i]);

    /* Illustrates interleaved free and live chunks, not an RSS benchmark.
       Pages are not explicitly touched; observed RSS may vary. */

    /* Request memory reclamation */
    malloc_trim(0);   /* try to release free pages; the result depends on block layout */

    return 0;
}

mmap vs brk — Why Large Allocations Are Different

If an allocation receives a separate mmap() mapping, free() can remove it through munmap(). A large request does not guarantee a separate mapping, and an additional arena is not the same as an individual mapped allocation.

Hence the counterintuitive effect: a program doing many large allocations may have more stable RSS than one with a million small ones, because the large ones return to the system immediately while the small ones stay trapped in the arena. The threshold is also dynamic — glibc increases it when it detects the program freeing large blocks, to avoid costly mmap/munmap cycles.

Arenas — Multithreaded Scaling

In a multithreaded program, if all threads competed for a single arena, the lock on it would be a bottleneck. glibc solves this with multiple arenas: the main arena (on brk) plus additional thread arenas (on mmap), each with its own lock.

# Experiment with the arena limit; compare RSS and throughput:
MALLOC_ARENA_MAX=2 ./program

# glibc statistics require function calls in the program:
# #include <malloc.h>
# malloc_stats();         /* writes to stderr */
# malloc_info(0, stdout); /* XML; also include <stdio.h> */

The trade-off is direct: more arenas = less lock contention, but more fragmented, unreturned memory (each arena holds its own free chunks). For multithreaded, RSS-sensitive applications, MALLOC_ARENA_MAX=2 is a common first optimization shot. This also explains why the same program uses more memory on a 64-core machine than a 4-core one.

Alternative Allocators — Measurement, Not Ideology

The following comparison is not a ranking. Results depend on allocator versions, object sizes, thread counts and allocation lifetimes.

The default ptmalloc2 in glibc is a general-purpose compromise. For specific workloads, specialized allocators offer a measurable advantage.

# Same program and data; the jemalloc path depends on the distribution.
/usr/bin/time -v ./server
/usr/bin/time -v env LD_PRELOAD=/usr/lib/libjemalloc.so.2 ./server

# Compare maximum RSS, execution time and latency.
# Repeat measurements and alternate run order.
# Alternatively, link with jemalloc at build time:
gcc program.c -ljemalloc -o program

LD_PRELOAD uses symbol interposition, with limitations explained in ELF and dynamic linking. Compare allocators on the same workload and warm-up period. A single RSS reduction does not establish the effect on latency or throughput.

Diagnostics — Tools

Conclusion: The Allocator as a Layer You Must Understand

malloc and free aren’t thin wrappers over the kernel — they’re a sophisticated memory management layer with arenas, bins, chunks, and anti-fragmentation strategies. An engineer who understands this layer doesn’t panic at high RSS after free(), knows when it’s fragmentation versus a real leak, and can deliberately match the allocator to the workload’s profile.

Compare RSS, process mappings and a profile of live allocations. Together they help distinguish allocator caches, fragmentation and leaks.

free() ends the program’s right to use a block, but does not promise lower RSS. Check who still owns the memory and which pages the allocator can return.

Related articles

• Anatomy of a Segfault — From MMU Through Kernel to gdb Core Dump — use-after-free and chunk corruption from the page-fault angle
• ELF and Dynamic Linking — What Happens Before main() Runs — LD_PRELOAD as the allocator swap mechanism
• mmap — When Memory-Mapped I/O Beats read() and write() — mmap for large anonymous allocations
• What fork() in Linux Really Does Under the Hood — copy-on-write and sharing heap pages

Technical references: mallopt(3), malloc_trim(3), malloc_stats(3).