Rate limiting that doesn't ruin your API: token bucket, sliding window, and honest 429s

Rate limiting has two jobs: protect your infrastructure from overload, and protect fair access for everyone else when one client misbehaves. A limiter that fires on legitimate burst traffic — a dashboard loading twelve widgets at once — is failing at the second job. Algorithm choice is what decides that.

The algorithms, honestly compared

• Fixed window: count requests per clock window (100/minute). Trivial to implement, but allows 2x burst at window boundaries - 100 requests at 11:59:59 and 100 more at 12:00:00 are both legal.
• Sliding window log: store every request timestamp, count the last 60s exactly. Perfectly accurate, memory cost grows with traffic - fine for low-volume expensive endpoints, wrong for high-QPS APIs.
• Sliding window counter: weighted blend of current and previous fixed windows. Accurate to within a few percent with O(1) memory - the pragmatic default for most APIs.
• Token bucket: tokens refill at a steady rate, requests spend them, bucket capacity allows controlled bursts. This models what users actually do (bursts followed by quiet) - it's what we use for user-facing limits.

A token bucket in 20 lines of Redis

// Atomic token bucket via Lua - one round trip, no races
const SCRIPT = `
local tokens = tonumber(redis.call('HGET', KEYS[1], 'tokens') or ARGV[1])
local ts = tonumber(redis.call('HGET', KEYS[1], 'ts') or 0)
local now = tonumber(ARGV[3])
local refilled = math.min(tonumber(ARGV[1]), tokens + (now - ts) * tonumber(ARGV[2]))
if refilled >= 1 then
  redis.call('HSET', KEYS[1], 'tokens', refilled - 1, 'ts', now)
  redis.call('EXPIRE', KEYS[1], 120)
  return {1, math.floor(refilled - 1)}
end
redis.call('HSET', KEYS[1], 'tokens', refilled, 'ts', now)
return {0, math.floor(refilled)}
`;

export async function allow(key: string, capacity = 20, perSecond = 5) {
  const [ok, remaining] = (await redis.eval(
    SCRIPT, 1, `rl:${key}`, capacity, perSecond, Date.now() / 1000,
  )) as [number, number];
  return { allowed: ok === 1, remaining };
}

The Lua script matters: a read-modify-write from application code has a race window under concurrency, and rate limiters are precisely the code path that gets hit concurrently. If you're on Cloudflare Workers or similar, Durable Objects give you the same single-writer guarantee without Redis.

Design the 429, don't just throw it

HTTP/1.1 429 Too Many Requests
Retry-After: 12
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 12

{ "error": "rate_limited", "retry_after_seconds": 12 }

Send RateLimit-* headers on every response, not just the 429 — well-behaved clients throttle themselves before hitting the wall. And send Retry-After with a real number; 'try again later' with no timing information guarantees the client retries immediately, which is the opposite of what you wanted.

Where to enforce

• Edge/gateway limits (per-IP, generous): shed abusive traffic before it touches your origin. This is DDoS hygiene, not fairness.
• Application limits (per-API-key or per-user): the fairness layer, where token bucket parameters map to your pricing tiers.
• Expensive-endpoint limits (per-operation): exports, searches, AI calls - one global limit either strangles cheap endpoints or fails to protect expensive ones.
• Never rate limit by IP alone for authenticated APIs - corporate NAT puts a thousand legitimate users behind one address.

The takeaway

Token bucket for user-facing fairness, sliding window counter for infrastructure protection, atomic storage for correctness, and honest headers so clients can cooperate. Rate limiting done well is invisible to legitimate users — that invisibility is the success metric.

Written by Appesto Engineering.