A rate limiter caps how many requests one client (a user, an API key, or an IP address) can make in a time window. It sits at the edge, before requests reach your backend, and protects you from abuse, runaway retries, and surprise cloud bills.

This question is less about drawing boxes and more about comparing algorithms and picking one with a reason. Interviewers use it to check whether you understand the difference between a textbook counter and something that survives 50 gateway instances running at once.

Requirements

Functional requirements

• Limit requests per identity (user ID, API key, or IP) to N requests per time window.
• Return 429 Too Many Requests with a Retry-After header when the limit is hit.
• Support different limits per plan (free vs. paid).

Non-functional requirements

• Accuracy costs latency: a perfectly precise limiter needs coordination between servers, which is slower. An approximate limiter is faster but may let a small burst through.
• The limiter check itself must add only single-digit milliseconds.
• It must work correctly across a fleet of stateless gateway instances, not just on one machine.
• It must fail in a predictable way if its counter store goes down. Usually fail open (let requests through) for availability, fail closed (reject) for security-critical endpoints.

Remember: a rate limiter that only works correctly on one server is not a rate limiter, it's a demo. The real question is always "does this hold up across a fleet?"

The five algorithms, and how to picture each one

Fixed window counter. Picture a whiteboard tally that gets erased every minute on the clock. Simple, O(1) memory, but a client can send N requests at 0:59 and another N at 1:00, getting 2N requests in two seconds right at the boundary.

Sliding window log. Picture keeping every receipt, then on each check throwing away receipts older than the window and counting what's left. Perfectly accurate, no boundary burst, but memory grows with request volume and each check costs more (you're storing every timestamp).

Sliding window counter. Picture two overlapping buckets: last window's count and this window's count, blended by how much of the previous window still overlaps "now." estimated_count = current_window_count + previous_window_count × (1 - elapsed_fraction_of_current_window) Near-accurate, O(1) memory, cheap to compute. This is the industry-favored answer (Cloudflare and Stripe-style gateways use it) because it balances accuracy and cost.

Token bucket. Picture a prepaid wallet. Every request spends one credit; credits refill at a steady rate up to a cap. This naturally allows a burst up to the bucket's size while still capping sustained abuse, which matches how real APIs behave: a page load firing 10 requests at once is fine, sustained hammering isn't.

Leaky bucket. Picture a queue draining at a constant speed. Requests join the back of the queue; if the queue is full, new ones are dropped. This smooths bursty traffic into a steady outflow, useful when the thing behind you can't handle spikes at all, but legitimate bursts get delayed instead of allowed.

Token bucket and leaky bucket are mirror images: one grants credit that drains over time, the other drains a fixed-rate queue in the opposite direction. If you can explain one, you can derive the other on a whiteboard.

The interview-favored pick: token bucket, or the sliding window counter, for API gateways. Bursts are normal client behavior, and you want to allow that while still capping sustained abuse.

Remember: token bucket lets bursts through and caps the average; fixed window is simple but leaks double the limit at the boundary.

Where to store counters: Redis vs. in-memory

In-memory only works if you pin a client to one gateway instance forever, which is fragile and breaks the moment you scale up or down. For a real distributed gateway, use Redis: INCR key with EXPIRE, or a Lua script so the check-and-increment happens in one atomic round trip. That avoids a race where two concurrent requests both see "under limit" before either one increments.

The Redis primitive follows the algorithm:

• Fixed window: INCR ratelimit:{key}:{window}, with EXPIRE set only on the first increment. The key self-cleans via TTL.
• Sliding window log: ZADD the timestamp into a sorted set, ZREMRANGEBYSCORE to drop anything older than now - window, then ZCARD to count. Wrap all three in one Lua script so nothing interleaves.
• Sliding window counter: two plain INCR keys, one per window. Read both, apply the weighted formula above. No Lua needed for reads.

API

High-level design

The limiter is middleware, not a separate network hop. It runs inline in the gateway's request path, calls Redis synchronously before forwarding the request, and Redis is the one shared truth every gateway instance checks. TTLs on every key mean old counters clean themselves up.

Deep dives

What breaks when you go distributed?

• Race conditions. Two concurrent requests from the same client can both read "count = 9, limit = 10" before either writes, letting both through. Fix it with an atomic Redis operation (INCR, or a Lua script) instead of a separate read then write.
• Redis as a single point of failure. If Redis goes down, decide up front: fail open (allow everything, risk overload) or fail closed (reject everything, risk false 429s for real users). Most production gateways fail open and lean on other safeguards, like autoscaling and circuit breakers, as a backstop.
• Clock skew across regions. Wall-clock-based windows can drift slightly when gateway instances span regions. Sliding window algorithms tolerate this better than fixed windows.
• Multi-region counters. A true global limit across regions needs either one global Redis (adds cross-region latency) or per-region limits that roughly sum to the global target (simpler, eventually consistent, usually good enough).

Trade-offs and follow-up questions

Sharding counters across a Redis Cluster scales horizontally, but one very abusive client still hammers a single shard. That's usually fine, since it's one client's traffic, not global traffic.

Add a local "fast reject" cache: if a client is already known to be far over the limit, reject it without even calling Redis, saving a network hop on the abuse case. Also return rate-limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) so well-behaved clients can throttle themselves before hitting the wall.

Q: How do you avoid a race condition when two requests arrive at nearly the same instant? A: Use an atomic operation: Redis INCR or a Lua script that reads, checks, and writes in one round trip, so nothing can interleave.

Q: What happens if Redis is temporarily unreachable? A: Decide and document the failure mode up front. Fail-open protects availability but risks abuse during the outage; fail-closed protects backends but causes false rate-limit errors. Most gateways fail open and rely on other safeguards.

Q: How would you rate-limit anonymous IP traffic differently from authenticated API-key traffic? A: Different key prefixes and different configs (ratelimit:ip:{ip} vs. ratelimit:key:{api_key}), typically with the stricter IP-based limit applied first at the edge, and the more generous authenticated limit enforced deeper in the gateway.