# Stream changes with CDC

> Read the Database's write log and publish each change to a stream, so search and other copies follow the data without a second write.

Publish every change from the write log to an Event Stream, and let consumers build their copies from it.

## Move up when

- The app writes to the Database and the search index in one request, and the two drift apart.
- A warehouse, a cache, or another service needs to follow the data.
- Stale search results appear after a crash between the two writes.

## The Design



The App Server writes once, to the Database. The CDC Connector reads the Database's log and publishes each change to the Event Stream. A Worker reads the stream, which in a real system is where the search index gets updated. The Search Engine here only answers the App Server's queries, because a Worker in the game cannot link to it. The stream is **Keyed by entity**, so a row's changes stay in one partition and arrive in order.

## Why it works

The app never writes the same fact twice, so a crash cannot leave the copies disagreeing for good. The connector reads the log, so it adds no work to a write. The stream keeps its messages, so a new index can replay them, and a second consumer can read them too.

Watch consumer lag, not the connector. A slow index shows up as growing lag, and the fix is in the consumer or the stream.

## What it costs

You add a connector, a stream, and a Worker, and each has its own Burn and failure modes. The copy is eventually consistent, so a search can miss the newest row for a moment. A consumer further behind than the retention window loses the oldest changes. An unkeyed stream with several partitions can apply an old change after a newer one.

## In Quorum

The [CDC Connector](https://wiki.playquorum.dev/components/cdc.md) needs [Change data capture](https://wiki.playquorum.dev/research/cdc.md), which follows [Event streaming](https://wiki.playquorum.dev/research/event-streaming.md) and [Data warehousing](https://wiki.playquorum.dev/research/data-warehouse.md). Place it beside the [Event Stream](https://wiki.playquorum.dev/components/log.md). The pattern is [Change data capture](https://wiki.playquorum.dev/patterns/change-data-capture.md), and the older option is the [Transactional outbox](https://wiki.playquorum.dev/patterns/transactional-outbox.md). The glossary has [CDC](https://wiki.playquorum.dev/glossary/cdc-change-data-capture.md). The next rung moves reports off the primary: [Reports off the primary, in a warehouse](https://wiki.playquorum.dev/patterns/reports-off-the-primary-warehouse.md).

Source: https://wiki.playquorum.dev/patterns/stream-changes-with-cdc