Articles

What Is a Distributed Lock?

A distributed lock coordinates exclusive access to a shared resource across multiple processes or machines, preventing race conditions in distributed systems.

The Lycoris Team The Lycoris Team · · 5 min read
A network switch with multiple cables connected

A distributed lock is a coordination mechanism that lets multiple independent processes — possibly running on different machines with no shared memory — agree that only one of them may hold access to a given resource at a time. It’s the distributed-systems equivalent of a mutex, solving the same problem a single-process lock solves, but without the luxury of shared memory to make acquiring and releasing it trivially safe.

Why a local mutex isn’t enough

Inside a single process, a mutex works because the operating system guarantees atomic operations on shared memory — two threads can’t both believe they hold the lock at the same instant, because the lock itself lives in memory both threads can see and the hardware enforces atomicity on the operations that check and set it. That guarantee evaporates the moment “multiple workers” means multiple machines. A race condition that a mutex prevents on one machine can still happen across machines — two API servers processing the same request, two cron-triggered workers picking up the same job, two replicas both deciding they’re responsible for the same task — because there’s no shared memory left for a simple flag to live in.

A distributed lock solves this by moving the lock’s state to a system all the contending processes can reach over the network — commonly a database, Redis, or a coordination service — and using that system’s own consistency guarantees to make acquiring the lock atomic from every client’s point of view.

How a distributed lock actually works

Most distributed locks share a common shape, regardless of backend:

  1. Acquire. A client attempts to write a lock record — a key, an owner ID, and usually an expiration — using an atomic “set if not already held” operation. If the write succeeds, the client holds the lock; if it fails because another client’s record is already there, it doesn’t.
  2. Lease with a TTL. Because a process can crash, get network-partitioned, or simply hang while holding a lock, distributed locks are almost always issued as a lease with a time-to-live rather than held indefinitely. If the holder doesn’t renew it before the TTL expires, the lock is released automatically, so a dead process can’t wedge the resource forever.
  3. Release. The holder deletes its lock record when done, ideally checking that the record still belongs to it before deleting — otherwise a client whose lease already expired could accidentally delete a different client’s newer lock on the same key.
  4. Fencing. Because a lease can expire while the original holder is still, unknowingly, doing work — a long garbage-collection pause is the textbook example — a well-designed distributed lock issues a monotonically increasing fencing token with each acquisition. Downstream systems that receive a write from a lock holder can reject any write carrying an older token than one they’ve already seen, closing the window where a “zombie” holder that thinks it still owns the lock can cause damage after its lease has actually expired and been reassigned.

That fencing step is easy to skip and is exactly the detail that separates a distributed lock that’s correct under real failure conditions from one that merely works in the common case.

Common implementations

BackendHow locking worksBest fit
Redis (e.g. SET key value NX PX ttl)Single atomic command sets the key only if absent, with a TTLLow-latency locks where brief unsafety under rare failure scenarios is acceptable
ZooKeeper / etcdEphemeral nodes plus a consensus protocol (Raft or ZAB) guarantee a single leader for a lock, with session-based expiry tied to a heartbeatCorrectness-critical coordination, since the underlying consensus algorithm avoids split-brain by construction
Relational database row lock or advisory lockA row insert or database-native advisory lock function acts as the lock record, with the database’s own atomicity backing itSystems that already depend on the database for correctness and want one fewer moving part

Single-node Redis is the fastest and simplest option, but a naive implementation is vulnerable if that Redis node fails over — the replica taking over may not yet have replicated the lock key, letting a second client acquire it while the first still believes it holds it. Multi-node algorithms built specifically to close that gap exist, but they add complexity. Systems built on Raft-based coordination services avoid the problem structurally, at the cost of higher latency per lock operation, because they wait for a quorum before confirming a lock is held — the same trade Raft makes generally, exchanging some speed for a stronger consistency guarantee, a tradeoff also explored in the CAP theorem.

The dangers of getting it wrong

Distributed locks are deceptively easy to implement unsafely. The failure modes that catch teams out most often:

  • Clock drift and pauses. A lock’s TTL is a wall-clock assumption, and any large, unexpected pause — a long GC cycle, a suspended container, a slow disk write — can make a client believe it still holds a lock whose lease already expired elsewhere. Fencing tokens exist specifically to make this survivable.
  • Split brain. If the lock backend itself isn’t strongly consistent, two nodes of that backend can each believe they’re authoritative and grant the same lock to two different clients simultaneously. This is why coordination services built on real consensus algorithms are preferred for locks where correctness genuinely matters.
  • Assuming locks are free. Every lock acquisition is a network round trip, and a system that reaches for a distributed lock on every request instead of only where genuine cross-process coordination is required will pay for it in latency and in Redis or database load that scales with traffic rather than with actual contention.

When you actually need one

Not every case of “two things might run at once” needs a distributed lock. If idempotency is achievable — designing an operation so that running it twice produces the same result as running it once — that’s usually simpler and more robust than coordinating exclusive access up front. Distributed locks earn their complexity when an operation genuinely can’t be made safe to run twice: allocating a scarce, non-idempotent resource, ensuring exactly one instance of a scheduled job runs per interval, or serializing writes to a piece of state that a database’s own transaction isolation doesn’t already protect.

The takeaway

A distributed lock extends the mutex pattern across machines that don’t share memory, using an external, network-reachable system — Redis, a consensus-backed coordination service, or a database — as the single source of truth for who currently holds the lock. The mechanism is simple in the common case, but correctness under real failures depends on details that are easy to skip: TTL-based leases so a crashed holder doesn’t wedge the resource, and fencing tokens so a lock holder that’s outlived its lease can’t corrupt state after the fact. Reach for one only when the operation genuinely can’t be made idempotent instead.

Chisato Chisato · · 4 min read

Byzantine Fault Tolerance Explained

Byzantine fault tolerance lets a distributed system keep working correctly even when some nodes fail arbitrarily or actively lie, not just crash cleanly.

#Distributed Systems #Computer Science #Databases
The Lycoris Team The Lycoris Team · · 4 min read

What Is the Strangler Fig Pattern?

The strangler fig pattern replaces a legacy system incrementally by routing traffic to new services piece by piece, until nothing old remains.

#Backend #Distributed Systems #Software Architecture
Chisato Chisato · · 5 min read

Raft vs Paxos: Consensus Algorithms Compared

Raft and Paxos both let a distributed cluster agree on a value despite failures — Raft trades some flexibility for a design built to be understood.

#Distributed Systems #Computer Science #Databases