In Part 1, we learned that rate limiters shape traffic. In Part 2, we looked inside the box and saw how algorithms like Token Bucket and Leaky Bucket fail in real life.
So far, we’ve assumed one simple thing: your entire application runs on a single server.
That assumption is about to break.
In a real production environment, you don’t run a single application server. You run tens, hundreds, or thousands of instances behind a load balancer. The moment you scale out, picking the right rate-limiting algorithm stops being your hardest problem.
Your hardest problem becomes: Where does the count live, and who is allowed to trust it?
A team I worked with once spent two full weeks investigating why users were complaining about getting 429 Too Many Requests errors way too early.
The bug? Their “100 requests per minute” Token Bucket was stored in local application memory across 8 server instances.
Instead of enforcing a global limit of 100 requests per minute, they were actually allowing 800 requests per minute in total! But because traffic wasn’t split perfectly by the load balancer, unlucky users hitting Instance 2 got blocked at 60 requests, while lucky users hitting Instance 3 sailed past 500.
The rate limiter wasn’t lying about its math—it was lying about whether a single, unified limit even existed.
In the rest of this Deep Dive, you’ll learn:
Why naive Redis counters have a hidden race condition that only triggers under heavy traffic.
The exact Lua script trick that makes Redis rate limiting 100% thread safe.
How server clock drift across machines can quietly break your sliding windows.
Strict Global vs. Local Consistency: How to pick the right architecture without killing your API latency.
Subscribe to master Rate Limiting
This Deep Dive comes with a set of reusable engineering assets — designed to outlast the article itself.
Free:
Rate Limiter Decision Matrix — one-page lookup: traffic pattern → recommended algorithm → known failure mode to watch for.
Premium:
Rate Limiter Failure Mode Cheat Sheet — visual breakdown of each algorithm’s breaking point under load, with the production symptom to watch for.
C# Reference Implementations Pack — tested token bucket, sliding window, and leaky bucket implementations, including the atomic Redis-backed variants from this part.
ADR: “Choosing a Rate Limiting Strategy” — a filled-in Architecture Decision Record, worked through using this Deep Dive’s reasoning, ready to adapt to your own system.
Distributed Rate Limiter Diagram Vault — the race condition, clock skew, and queue-latency scenarios from this series, rebuilt step-by-step as reference diagrams.
(Available via the Gumroad Premium Library)
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.