Garbage Collection: Three Approaches and What Each Costs

Automatic memory management removes a category of bug and introduces a scheduling problem: the collector needs CPU and sometimes needs the program to stop.

For a separate people-operations perspective, the reference covers capacity and workflow analysis.

Every collector trades three things against each other — throughput, pause time, and memory footprint — and no design gives you all three. Knowing which one your collector prioritises explains most of its behaviour.

Reachability, and the two costs

The collector determines which objects are reachable from a set of roots — stack variables, registers, globals — and treats everything else as garbage.

This has an important consequence: an unreachable object is collected regardless of whether you meant to keep it, and a reachable object is never collected regardless of whether you want it. A "memory leak" in a managed language is almost always a reference you forgot about, usually in a collection that grows without bound.

Two costs, and only the second is the one people mean:

Collector CPU, which competes with your program's work.

Pauses, where the program stops so the collector can move objects or scan without interference. This is what shows up in tail latency.

The three families

Tracing collectors

Walk the object graph from the roots, mark what is reachable, reclaim the rest. Most mainstream managed runtimes.

Generational, almost universally, based on the empirical observation that most objects die young. The heap is split: new objects are allocated in a young generation collected frequently and cheaply; survivors are promoted to an old generation collected rarely and expensively.

This is why allocation is cheap in these runtimes. Allocation is a pointer bump in a contiguous region, and collecting the young generation only touches surviving objects rather than dead ones — so a young collection over mostly-dead objects is nearly free.

And why the advice to avoid allocation is often wrong. Short-lived allocation is what generational collectors are designed for. What is expensive is objects that survive long enough to be promoted and then die — you paid the copy and got no benefit.

Reference counting

Each object tracks how many references point at it; the count reaching zero frees it immediately.

Advantages: deterministic — memory is freed at a known point — and no pauses.

Costs: every reference assignment updates a counter, which is constant overhead spread through the program; concurrent updates need atomics, which are expensive; and cycles are never collected, because a group of objects referencing each other keeps everyone's count above zero.

Python uses reference counting with a cycle detector on top. Swift uses reference counting and requires the programmer to break cycles with weak references.

Concurrent and incremental collectors

Do most of the work while the program runs, reducing pauses at the cost of throughput and complexity. Modern low-latency collectors — ZGC, Shenandoah, Go's collector — aim for pauses in single-digit milliseconds largely independent of heap size.

The cost is real: more CPU overhead, more memory headroom required, and a rate problem. If the program allocates faster than the collector can concurrently reclaim, the collector falls behind and must eventually stop the program anyway — with a longer pause than a non-concurrent collector would have had.

Choosing, roughly

Throughput-oriented collectors maximise total work done, accepting longer pauses. Right for batch processing where total time matters and no user is waiting.

Low-latency collectors minimise pauses at some throughput cost. Right for interactive services where p99 latency is the metric.

Memory-constrained environments may need a collector that runs more often in less space, costing both throughput and latency.

The decision follows from what you are optimising. A batch job tuned for low pause times is wasting CPU it could have spent on the work.

Tuning, in the order worth trying

1. Measure before changing anything. Enable collection logging and look at pause frequency, pause duration, and how much is reclaimed each cycle. Almost every problem attributed to the collector is a problem with allocation behaviour.

2. Reduce allocation of medium-lived objects. Short-lived is fine. The expensive pattern is objects surviving the young generation and dying shortly after — you paid to copy them and gained nothing. Caches with short expiry are a classic source.

3. Size the heap. Too small means constant collection; too large means longer collections and, in a container, being killed. Watch resident memory against the container limit, not just heap usage — see what actually happens when you run out of memory.

4. Size the young generation. For a service allocating heavily and short-lived, a larger young generation means fewer promotions and fewer expensive old-generation cycles.

5. Only then change collector. And measure, because the low-latency choice is not free.

The symptoms and what they usually mean

Regular pauses correlated with allocation rate. Normal young-generation collection. If the pauses are acceptable, this is the collector working.

Long infrequent pauses. Old-generation collection. If they are unacceptable, either the old generation is too large or too much is being promoted.

Pauses growing over time, ending in a very long one. Something is retaining objects. Take a heap dump and look at what holds the largest retained set. Usually an unbounded collection — a cache, a listener list, a map keyed by something that keeps growing.

Collector CPU rising while reclaimed memory falls. The heap is filling with live objects. This is the pattern immediately before an out-of-memory failure, and it is a good thing to alert on.

Pauses in a concurrent collector. The concurrent phase could not keep up with allocation. Either allocate less or give the collector more headroom.

Interaction with containers

Set the heap from the container limit, with headroom. Non-heap memory — thread stacks, metaspace, direct buffers, the runtime itself — is substantial, and a heap sized to the container limit guarantees an eventual kill.

Verify the runtime sees the cgroup limit, not the host's. Modern runtimes are container-aware and it is version-dependent. See container isolation.

Collector threads default to the CPU count, which under a CPU quota means more threads than the container can run, producing contention. Check what your runtime believes it has.

What actually helps in application code

Bound every cache. The single most common cause of a managed-memory leak.

Remove listeners and callbacks. A registered callback holds a reference to everything it closes over.

Watch static and long-lived collections. Anything reachable from a static field lives forever by construction.

Stream rather than materialise. Reading a whole result set into a list makes memory proportional to the largest result you will ever see.

Reuse buffers on hot paths only, where you have measured that it matters. Object pooling as a general practice fights the collector's design and usually loses.

The summary

Throughput, pause time and footprint — pick two, and know which two your collector picked.

Short-lived allocation is cheap; medium-lived allocation is expensive. Optimising the wrong one is common.

Most "GC problems" are retention problems. Take the heap dump before changing collector flags.

In containers, size the heap from the limit with headroom, and verify the runtime can see the limit at all.