What Is Rate Limiting? Algorithms and When to Use It

Rate limiting is the practice of capping how many requests a client can make to a service within a given time window, and rejecting or delaying requests beyond that cap. It protects an API or server from being overwhelmed — whether by a genuine traffic spike, a buggy client stuck in a retry loop, or a deliberate abuse attempt like credential stuffing or scraping. Without it, a single misbehaving client can degrade service for everyone.

Why every public API needs one

An API without rate limits has no defense against volume. A client with a bug that retries too aggressively can flood a server with the same effect as an attack. A scraper can pull an entire dataset in minutes. A brute-force login attempt can try thousands of password guesses per second. Rate limiting doesn’t eliminate any of these, but it bounds the damage: a misbehaving client gets throttled instead of taking the service down for legitimate users. It’s frequently deployed alongside a load balancer, which distributes traffic across servers, and a web application firewall-style layer that filters malicious requests — rate limiting handles the volume problem specifically.

Common algorithms

Different rate-limiting algorithms make different tradeoffs between accuracy, memory use, and how they handle bursts.

Fixed window. Count requests in a fixed time slice — say, per calendar minute — and reset the counter when the slice ends. Simple to implement and cheap to store, but it has a boundary problem: a client can send its full quota in the last second of one window and again in the first second of the next, doubling the effective rate for a brief moment.

Sliding window. Instead of resetting at a hard boundary, the window continuously slides with the current time, smoothing out the boundary burst. It’s typically implemented by weighting the previous window’s count based on how much of it still overlaps the current moment, giving a good approximation without storing every individual timestamp.

Token bucket. Each client has a bucket that holds a fixed number of tokens, refilled at a steady rate. Every request consumes one token; if the bucket is empty, the request is rejected. This naturally allows short bursts (as long as tokens are available) while enforcing a steady average rate over time — it’s one of the most widely used algorithms because it matches how real traffic behaves.

Leaky bucket. Requests are added to a queue and processed at a constant, fixed rate, regardless of how bursty the incoming traffic is — like water leaking out of a bucket at a steady pace no matter how fast it’s poured in. This produces very smooth outbound traffic but adds latency for bursts, since excess requests wait in the queue rather than being processed immediately.

Algorithm	Handles bursts	Memory cost	Smoothing
Fixed window	Poorly (boundary spikes)	Low	None
Sliding window	Well	Moderate	Good
Token bucket	Well, by design	Low	Good
Leaky bucket	Not at all (queues instead)	Moderate	Best

Where to enforce it

Rate limits can live at several layers, and production systems often combine more than one:

• Edge/CDN layer. Blocking obvious abuse before it reaches application servers at all — cheapest place to stop volumetric attacks.
• API gateway. A common place to apply per-API-key or per-endpoint limits, often the layer that also handles CORS and authentication.
• Application layer. Fine-grained limits tied to business logic, like capping how many password reset emails one account can trigger per hour.

Identifying who to limit

A rate limit needs a key to count against — IP address, API key, user ID, or a combination. IP-based limiting is the simplest and works without authentication, but it’s imprecise: many users can share one IP behind NAT or a corporate proxy, and a determined attacker can rotate IPs. API-key or user-ID-based limiting is more accurate for authenticated traffic — the same key a service issues alongside a JWT for authorization is often reused as the rate-limit key, since it already uniquely identifies the caller.

Communicating limits to clients

A well-designed API tells clients where they stand rather than leaving them to guess. The de facto convention is a set of response headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1750000000

When a client exceeds its limit, the server should return HTTP status 429 Too Many Requests, ideally with a Retry-After header telling the client how long to wait before trying again. This gives well-behaved clients enough information to back off gracefully instead of hammering the server with immediate retries — a pattern that matters just as much for REST APIs as it does for webhook delivery, where the receiving side may need to signal that it’s overwhelmed.

The takeaway

Rate limiting bounds how much load any single client can put on a service, turning a potential outage into a throttled request. Token bucket is the most common algorithm because it allows natural bursts while holding a steady average; fixed window is the simplest but leaks accuracy at window boundaries. Pick a key that actually identifies the caller — API key or user ID over raw IP when you can — and always tell rejected clients when they can retry.