What Is a Load Balancer? How It Works, Explained

A load balancer is a component that sits in front of a group of servers and distributes incoming network traffic across them, so that no single server becomes a bottleneck or a single point of failure. It is one of the most fundamental building blocks of scalable, resilient web architecture — present in virtually every production system that needs to handle more traffic than one machine can serve.

The idea is simple. Instead of pointing users at one server, you point them at the load balancer, and it decides — connection by connection or request by request — which healthy backend should do the work. Everything else about load balancing (routing algorithms, health checks, TLS handling) is a refinement of that one job.

Layer 4 vs Layer 7

Load balancers operate at different levels of the networking stack, and the distinction matters.

Layer 4 (transport-layer) load balancers work at the TCP/UDP level. They route traffic based on IP address and port without inspecting the content of the packets. They are fast and low-overhead — the load balancer is essentially forwarding raw streams — but they have no visibility into HTTP headers, paths, or cookies.

Layer 7 (application-layer) load balancers understand the full HTTP request. They can route based on URL path (/api/* to one server pool, /static/* to another), HTTP headers, hostnames, or even request body content. This flexibility makes Layer 7 balancers far more common in modern web applications. Most cloud-managed load balancers — AWS ALB, Google Cloud Load Balancing, Azure Application Gateway — operate at Layer 7.

	Layer 4 (transport)	Layer 7 (application)
Sees	IP addresses, TCP/UDP ports	Full HTTP requests
Routes on	Address and port	Path, hostname, headers, cookies
Overhead	Minimal — forwards packets	Higher — parses every request
TLS	Usually passes encrypted traffic through	Terminates and inspects
Typical examples	AWS NLB, HAProxy in TCP mode	AWS ALB, Nginx, Envoy
Best for	Raw throughput, databases, non-HTTP protocols	Web apps, APIs, microservices

Neither layer is “better” — they solve different problems, and large systems often run both: a Layer 4 balancer soaking up raw connections at the edge, handing off to Layer 7 balancers that make smart routing decisions.

Routing algorithms

When multiple healthy backend servers are available, the load balancer uses a routing algorithm to pick one:

• Round-robin — requests are distributed sequentially, one server at a time, cycling back to the first. Simple and fair when servers are identical.
• Weighted round-robin — the same rotation, but more capable servers receive a proportionally larger share of traffic. Useful during migrations, when old and new instance types briefly coexist.
• Least connections — the request goes to whichever server currently has the fewest active connections. Better for workloads where requests vary widely in duration, since a server stuck on slow requests stops receiving new ones.
• Least response time — a refinement that also factors in how quickly each backend has been responding, steering traffic away from degraded servers before they fail outright.
• IP hash / consistent hashing — the client’s IP address (or another key) is hashed to always map to the same backend. Useful when some server-side state must stay on the same machine.

Consistent hashing deserves a special note because it powers most distributed caches. With a naive hash (server = hash(key) % N), adding or removing one server remaps almost every key. Consistent hashing arranges servers on a ring so that changing the pool size only remaps a small fraction of keys — which means a scaling event doesn’t wipe out your cache hit rate.

Health checks and failover

A load balancer continuously checks whether its backends are healthy, typically by sending a simple HTTP request to a /health endpoint every few seconds. If a server fails to respond (or returns an error), the load balancer removes it from rotation. Traffic automatically shifts to the remaining healthy servers without manual intervention. When the server recovers, it is added back.

Checks come in two flavors. Active checks are the probes described above — synthetic requests on a schedule. Passive checks watch real traffic instead: if a backend starts returning errors or timing out on live requests, it gets ejected without waiting for the next probe. Mature setups use both, plus connection draining — when a server is being removed (for a deploy, say), the balancer stops sending it new requests but lets in-flight ones finish, which is what makes zero-downtime deployments possible.

This automatic failover is why a load balancer dramatically improves availability. Without it, a crashed backend would cause roughly 1/N of requests to fail, where N is the number of servers — and users would see intermittent errors with no obvious cause.

Sticky sessions

Some applications store per-user state in server memory (shopping carts, session tokens). Sticky sessions (also called session affinity) configure the load balancer to always route a given client to the same backend, typically tracked via a cookie. This preserves server-side state across requests. It is a useful workaround but comes with a trade-off: if the sticky server goes down, the session is lost anyway. Stateless architectures — storing session data in a shared store like Redis — are generally preferable for resilience, and they let any backend serve any request.

TLS termination

Establishing a TLS connection is computationally expensive. TLS termination at the load balancer means the encrypted connection from the client ends at the load balancer, which decrypts the request and forwards plain HTTP to the backend servers. The backends only need to handle unencrypted traffic internally, simplifying their configuration and concentrating certificate management in one place. Understanding TLS in this context pairs well with a grounding in how HTTPS works.

In stricter environments the balancer re-encrypts traffic to the backends instead of forwarding plaintext — a pattern that zero trust architectures treat as the default, since they refuse to treat the internal network as safe.

Where load balancing happens in a modern stack

“The load balancer” is rarely one box. A single request to a modern application typically passes through several layers of load balancing, each operating at a different scope:

• DNS. DNS can return multiple A records for a hostname, or different answers by region (GeoDNS), spreading users across data centers before a connection even opens. It is coarse — DNS caching makes failover slow — but it is the first hop.
• Anycast and the CDN edge. A CDN announces the same IP address from dozens of locations, and internet routing delivers each user to the nearest one. That is load balancing done by the network itself.
• The cloud load balancer. The managed Layer 7 (or Layer 4) balancer in front of your origin servers — the piece most people mean by the term.
• Kubernetes Services. Inside a Kubernetes cluster, a Service spreads traffic across the healthy pods behind it, with an Ingress or Gateway doing Layer 7 routing at the cluster boundary.
• Service meshes. In a microservices architecture, sidecar proxies balance every service-to-service call on the client side, with per-request retries and failover.

Serverless platforms bundle all of this — the provider’s infrastructure routes each invocation to available capacity, and you never see the balancer at all.

Common questions

Is a load balancer hardware or software? Historically it was a hardware appliance (F5 BIG-IP was the classic example). Today the overwhelming majority are software — Nginx, HAProxy, Envoy — or a managed cloud service running that software for you. The concepts are identical either way.

What is the difference between a load balancer and a reverse proxy? They overlap heavily. A reverse proxy is any intermediary that accepts requests on behalf of backend servers, and it may also cache, compress, or terminate TLS. “Load balancer” emphasizes one specific function: distributing traffic across multiple backends. Most modern tools are both at once — Nginx is the standard example.

Do I need a load balancer for a small site? With one server and no high-availability requirement, no. The moment you run two instances — for redundancy or for zero-downtime deploys — you need something to split traffic between them, even if it is just your platform’s built-in routing.

What happens if the load balancer itself fails? It would become the new single point of failure, so production setups run redundant balancer pairs with automatic failover, use anycast so another site absorbs the traffic, or lean on managed services whose redundancy is the provider’s problem. That last option is a big part of why managed load balancers are the default choice.

The takeaway

A load balancer distributes traffic across servers to eliminate single points of failure and enable horizontal scaling. Layer 7 balancers understand HTTP and enable intelligent routing; health checks provide automatic failover; TLS termination centralizes certificate management; and in a modern stack, balancing happens at several layers at once — DNS, CDN edge, cloud balancer, and cluster. For any service that needs to handle meaningful traffic or stay available through server failures, a load balancer is not optional infrastructure — it is the foundation.