CDC Connector

The CDC Connector publishes every change a Database makes as an event on a stream, as Debezium does.

When to place it#

• Place it when a second copy of the data has to follow the Database: a search index, a warehouse, or a feed for another service. It unlocks with Change data capture in Series B, for 18,000 ◇, after Event streaming and Data warehousing. It sits in Series B because it needs both things it feeds: a stream to publish into and a consumer worth feeding.

• Copy it when you add a second Database, since each Database needs its own connector.

How it works#

Debezium is a change data capture tool that reads a database's write-ahead log and publishes every insert, update and delete as an event. Link a Database into the connector, and the connector into an Event Stream. It takes a copy of the writes the Database processed, one change each, and publishes them to the stream as tasks, at 20,000 changes a second. Nothing else in the Design sees the copy, so the App Servers' writes still go to the Database and are still answered by it.

That is what makes it useful. The Search Engine, the Data Warehouse and a cache can each read the stream and follow the Database, and the app never writes the same fact twice. The alternative is a dual write: the app writes to the Database and to the index in one request, and a crash between the two leaves them disagreeing forever.

It carries no traffic of its own and has no options. Its settings are the Database it reads and the stream it publishes to. It never slows a write.

How it fails#

• Lag. The changes reach the stream, and a consumer reads the stream at its own pace. A search result or a report is behind by the stream's lag, and a consumer further behind than the retention window loses the oldest changes. The connector's card shows that lag, because the connector buffers nothing: the stream holds what has not been read.

• Order. A change is a copy, so it carries no ordering beyond the order the writes committed in. A stream with several partitions keeps order only inside a partition, so an unkeyed stream lets a superseded write land after the one that replaced it. Key the stream by the row's primary key.

• Its own rate. Past 20,000 changes a second it drops what it cannot publish, like any flat-capacity Component. A busy Database outruns one connector.

• No consumer. With nothing reading the stream, the changes it publishes are stranded.

• A loop. A consumer that writes back to the Database, with the stream reading that Database, closes a loop the board refuses: a change that causes a write that causes a change.

Instance failure#

• Signs: The CDC Connector card goes grey and its callout reads "The host under this VM died. It still bills." Requests sent to it are stranded.

• Do now: Select the Node and click Replace. It costs Cash and comes back cold.

• Prevent: Run a second Node of the Component, so one is still serving while the other is replaced.

Zone outage#

• Signs: Every Node in one zone goes grey at once, the CDC Connector with them.

• Do now: Nothing brings the zone back sooner. Fail over promotes a Standby, and this Component keeps none, so it ends on its own after at most 120 s.

• Prevent: Research Multiple zones and keep a copy of the Component in a second zone.

Region outage#

• Signs: One region's board goes grey and the CDC Connector in it goes down.

• Do now: Press Fail over on the banner. It needs Multiple zones and a second open region, and starts the Node again in the nearest other region, cold, for the price of a replacement. Otherwise it comes back when the incident ends: 180 sim seconds in the catalog, 60 once the error budget counts.

• Prevent: Research Multiple regions, keep a copy in a second region, and add GeoDNS to send users there.

Tips#

• Place it next to the Database it reads and the stream it writes.

• Read the lag, not the connector's latency.

• Look for a slow design's fix in the consumer or the stream, not the connector.