Articles

The Pub/Sub Pattern Explained

Publish-subscribe decouples senders from receivers through a message broker, letting services communicate without knowing who's listening.

Chisato Chisato · · 5 min read
Abstract blue network mesh

The publish-subscribe pattern, or pub/sub, is a messaging model where senders (publishers) emit events to a named channel without knowing who, if anyone, is listening, and receivers (subscribers) register interest in a channel without knowing who’s publishing to it. A broker sits in between, routing each published message to every current subscriber. Neither side holds a reference to the other — that decoupling is the entire point.

Why decoupling matters

In a direct request model, a service calling another service needs to know its address, be available at the same time, and wait for (or at least trigger) a response. Pub/sub removes all three requirements. A publisher can emit an “order placed” event and move on immediately; it doesn’t know or care whether zero, one, or ten different services react to it, and none of those subscribers need to exist yet at the time the message is published, depending on the broker’s delivery guarantees. Adding a new subscriber — say, a fraud-detection service that now also wants to see “order placed” events — requires no change to the publisher at all.

This is the architectural pattern behind event-driven microservices: services react to what happened elsewhere in the system rather than being explicitly called.

Topics and fan-out

Messages are published to a topic (also called a channel or subject) rather than to a specific recipient. Any number of subscribers can register on the same topic, and the broker delivers a copy of each message to every one of them — this is fan-out, and it’s the key difference from a queue.

Pub/sub vs a message queue

It’s easy to conflate pub/sub with a message queue, since both involve a broker sitting between producers and consumers, but they solve different delivery problems:

Message queuePub/sub
DeliveryEach message goes to exactly one consumerEach message goes to every subscriber
Typical useDistributing work across a pool of workersBroadcasting an event to multiple interested services
Consumer independenceConsumers compete for messagesConsumers each get their own copy
Losing a subscriberAnother worker picks up the messageThat subscriber simply misses it (unless persisted)

Some systems blend both: Kafka, for instance, is a log-based system where multiple independent consumer groups can each read the same topic in pub/sub fashion, while within a single consumer group, partitions are divided up queue-style so each message is processed once per group. Redis offers both a queue-like list structure and true pub/sub channels as separate primitives — see Redis vs Memcached for more on where Redis fits generally.

Delivery guarantees are not automatic

“The broker delivers messages to subscribers” hides a lot of nuance that varies by implementation:

  • At-most-once vs at-least-once — does a subscriber that’s offline when a message is published miss it entirely, or does the broker retain and redeliver it once the subscriber reconnects? Simple in-memory pub/sub (like Redis’s native PUBLISH/SUBSCRIBE) is fire-and-forget: if nobody is subscribed at publish time, the message is gone. Log-based systems like Kafka retain messages so late or reconnecting consumers can catch up.
  • Ordering — whether messages arrive in the order they were published, globally or just per-topic-partition, depends entirely on the broker’s design.
  • Exactly-once delivery is the hardest guarantee to make true end-to-end and usually means “at-least-once delivery plus idempotent handling on the consumer side,” not a magic property of the broker alone.

Don’t assume a guarantee your broker doesn’t actually document — this is one of the more common sources of subtle data loss in event-driven systems.

Where pub/sub shows up

  • Webhooks are a pub/sub pattern at the HTTP layer: a service “publishes” an event by POSTing to every URL that’s registered interest, though see webhooks explained for how that differs from broker-mediated pub/sub in practice.
  • Real-time UI updates, like a live dashboard or chat app, often use Server-Sent Events or WebSockets as the transport that carries a server-side pub/sub event out to a browser.
  • Cache invalidation across a fleet of servers is a classic pub/sub use case: one instance publishes “this key changed,” and every other instance subscribed to that channel evicts it locally.
  • Load-balanced services reacting to configuration changes, feature flag updates, or shutdown signals broadcast across a cluster, rather than each instance polling a central store — see load balancers for the layer this usually sits alongside.

Scaling subscribers without touching publishers

The practical payoff of this decoupling shows up most clearly when a system grows. Adding a new consumer of an existing event stream — a new analytics pipeline that wants to see every “user signed up” event, say — is a matter of registering a new subscriber on an existing topic, with no code change and no coordination required on the publishing side. Compare that to a direct-call architecture, where adding a new downstream consumer means finding every place the original event is triggered and adding another call, with the risk of missing one. This is a large part of why pub/sub shows up so often in systems built around microservices rather than a monolith: independent teams can each subscribe to events they care about without needing write access to, or even visibility into, the service that originally publishes them.

The same property that makes this convenient also makes it easy to lose track of who’s actually listening to a given topic — since publishers have no visibility into their subscribers, a topic can accumulate consumers over time that nobody remembers exist, which is worth keeping in mind when planning to change or retire an event’s shape.

The takeaway

Pub/sub decouples who sends a message from who receives it: publishers don’t address specific subscribers, and subscribers don’t know who’s publishing, with a broker handling routing and fan-out to every current subscriber on a topic. It’s the right pattern when multiple, independently-evolving parts of a system need to react to the same event without hard-coding a dependency on each other. Just don’t assume delivery, ordering, or exactly-once semantics beyond what your specific broker actually guarantees — those details vary widely and are worth checking before you depend on them.

Chisato Chisato · · 5 min read

Exponential Backoff and Retry Strategies Explained

Exponential backoff spaces retries further apart after each failure so clients stop hammering a struggling service. How it works, and why it needs jitter.

#DevOps #Cloud #Distributed Systems
Chisato Chisato · · 4 min read

Active-Active vs Active-Passive Architecture

Active-active runs every region live and load-balanced; active-passive keeps a standby idle until failover. How each affects cost, consistency, and recovery.

#Cloud #DevOps #Distributed Systems