What Is Change Data Capture (CDC)? Explained

Change data capture, or CDC, is a technique for identifying and streaming row-level changes — inserts, updates, deletes — out of a database as they happen, instead of periodically querying the whole table to see what’s different. It’s the mechanism behind keeping a search index, a cache, a data warehouse, or another service’s local copy of data in sync with a source database in near real time.

The problem with polling

The naive way to notice database changes is polling: run a query every few minutes that checks for rows updated since the last check, using a updated_at timestamp column or similar. This works for simple cases but has real limitations. Polling misses deletes entirely, unless you maintain a separate tombstone mechanism, since a deleted row simply isn’t there to query anymore. It also misses intermediate states — if a row is updated three times between polls, you only see the final value, which is fine for some use cases and wrong for others, like an audit log that needs every change. And it introduces latency equal to the polling interval, plus load on the source database from repeated full or filtered scans.

CDC solves all three problems by capturing every change as it commits, in order, without querying the table itself.

How CDC actually captures changes

The most common and most reliable approach reads the database’s write-ahead log (also called a transaction log, binlog, or WAL depending on the database) — the internal, append-only record every transactional database already maintains to support crash recovery and replication. Every insert, update, and delete gets recorded there before it’s considered committed, which makes it a complete, ordered, low-overhead source of truth for what changed and when.

Log-based CDC tools (such as Debezium, or built-in features in some managed database services) tail this log continuously and translate each entry into a structured change event, typically containing the row’s primary key, the operation type, and the before/after values. Because it reads the log rather than querying tables, it adds minimal load to the source database and captures every intermediate state, including deletes.

Other, less common approaches include trigger-based CDC (database triggers write change records to a separate table on every write, adding overhead to every transaction) and timestamp-based polling (the naive approach described above, sometimes used when log access isn’t available). Log-based CDC is generally preferred where it’s supported, precisely because it avoids adding load to the write path.

Where the change stream goes

A CDC stream on its own is just a sequence of change events — the value comes from what consumes it. Common destinations:

• A message queue or streaming platform like Kafka, which decouples the source database from downstream consumers and lets multiple systems subscribe to the same change stream independently. This composes with the broader pattern of using a message queue to decouple producers from consumers generally.
• A search index, kept in sync so that a full-text search engine reflects the current state of the source data without needing to be rebuilt from scratch.
• A cache, invalidated or updated as the underlying row changes, instead of relying purely on time-based expiration.
• A data warehouse, where CDC feeds an OLAP system that’s optimized for analytical queries the source OLTP database isn’t built to handle efficiently.
• A materialized view in a different system entirely, kept current by replaying the change stream rather than periodically re-running the defining query. See what a materialized view is for the same idea applied within a single database.

CDC vs traditional replication

CDC and database replication are closely related but serve different purposes. Native database replication typically keeps a full standby copy of the same database, for failover and read scaling, using the same schema and often the same database engine. CDC is more general-purpose: the change stream can feed a completely different kind of system — a search index, a document store, an analytics warehouse — with a different schema, different technology, and a transformation step in between. Some CDC implementations are literally built on top of the same replication protocol a database uses for its own standbys, just consumed by an external tool instead of another instance of the same database.

Ordering, exactly-once, and idempotency

Because downstream consumers apply changes asynchronously, two guarantees matter for correctness: changes for a given row need to be applied in the order they occurred (an update followed by a delete must not be applied in reverse), and consumers need to handle the possibility of receiving the same event more than once, since most streaming systems guarantee at-least-once delivery rather than exactly-once. This makes idempotency essential on the consuming side — applying the same change event twice should produce the same end state as applying it once, typically by keying updates on the source row’s primary key and a monotonically increasing sequence number from the log.

The takeaway

Change data capture streams row-level database changes out in near real time by reading the database’s own write-ahead log, avoiding the latency, missed deletes, and repeated-scan overhead of polling for changes. The resulting stream typically flows through a message queue to feed search indexes, caches, or analytics warehouses, decoupling the source database from whatever systems need to stay in sync with it. Because delivery is usually at-least-once rather than exactly-once, consumers need to apply changes idempotently to stay correct.