Captura de cambios de datos
Lee el registro de cambios de la base de datos y publica cada cambio como un evento, para que el buscador, el almacén de datos y la caché sigan al día sin que la app escriba dos veces.
Change data capture turns the database's own log into a stream of events that other systems follow.
Use it when#
A second copy of the data must follow the database: a search index, a warehouse, a cache, or another service.
The app currently writes to two stores, and they drift apart after a crash.
You cannot change every writer to emit events.
How it works#
Every database writes changes to a log before it applies them. A connector such as Debezium reads that log and publishes each insert, update, and delete to a stream. Consumers read the stream and apply the changes to their copies. The app writes once, to the database, so there is nothing to disagree.
Key the stream by the row's primary key. Order holds only inside a partition, so an unkeyed stream can apply an old write after the newer one.
What it costs#
Copies lag the database by however far behind the consumer is, and a consumer further behind than the stream's retention loses changes. Changes are row-level, so consumers see the schema, and a migration can break them. You run a connector and a stream. A consumer that writes back to the source can start a loop.
In Quorum#
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| App Server | Instance size | medium | Medium, so the App Server is not the limit. |
| Database | Instance size | medium | Medium. The app writes here once, and the connector reads its log. |
| Event Stream | Partition key | On | Keyed by the row, so changes to one row apply in the order they committed. |
| Event Stream | Partitions | 3 | Three partitions allow up to three consumers. This Design links the Search Engine, so its read rate sets the pace. |
Research this Design needs: Full-text search, Change data capture, Event streaming, WebSockets, Message queues, Full-text search, Event streaming, Change data capture.
Link the Database into a CDC Connector, the connector into an Event Stream, and the stream into the copy that follows: a Search Engine, a Data Warehouse, or a Cache. The connector has no options of its own. It copies the writes the Database processes and never slows them. The stream's key and partitions set order and read speed.
The Search Engine in the picture also answers the App Server's queries; the other extra Nodes only serve the remaining Series B request kinds. Research Change data capture, which needs Event streaming and Data warehousing. The same effect without a stream is the transactional outbox. The glossary defines CDC.