# Event Stream

> A log, not a queue: events stay after they are read, so a consumer can replay them. Consumers read one partition each and fall behind at their own pace.

The Event Stream stands for Apache Kafka, Redpanda and Amazon Kinesis: a partitioned, replicated **log** that keeps messages after they are read.

## When to place it

- Place it when more than one consumer needs the same events, such as a search index, a warehouse, or a second service reading what the first one acted on.
- Place it when the events must survive. It unlocks with [Event streaming](https://wiki.playquorum.dev/research/event-streaming.md) in Series B, for 15,000 ◇, after Message queues.

## How it works

Producers append, consumers read at their own pace, and the messages stay after they are read. That separates it from a [Queue](https://wiki.playquorum.dev/components/queue.md). A Queue hands a task to one Worker and forgets it. A log keeps it for the retention window, so a consumer can read it twice, a second consumer can read it at all, and a new index or a rebuilt Worker can replay it without touching the [Database](https://wiki.playquorum.dev/components/database.md).

It takes `task` from the Nodes that produce async work and sends `task` on to the Workers that consume it. One broker appends **100,000 messages a second** at its first size, and the rate doubles at each of its six sizes. Each append waits five milliseconds times its acknowledgement setting.

**Consumer lag** is the stat to watch: the events a stream holds that its consumers have not read. It is the log's version of queue depth. A producer never waits for its consumers, so the first harm of a slow Worker is a growing log, not a slow request.

## Options

| Option | What it decides |
| --- | --- |
| Partitions | How many consumers can read at once. A consumer group uses one consumer per partition, so a stream reads at most one partition's rate per partition: 20,000 messages a second each. |
| Partition key | Order. A key keeps one entity's events in one partition, so they stay in order, but one key with all the traffic makes a hot partition. Round robin spreads evenly and loses order. |
| Replication factor | Safety. Each replica is another copy of every partition, and bills half a broker's Burn again. With one replica a broker that fails takes every event it held; with more, the acknowledgements decide what is lost. |
| Acknowledgements | Latency. `0` does not wait, `1` waits for the partition leader, `all` for every replica. |
| Retention | How long a message stays. Ten hours, two and a half days, or a week. |

## How it fails

- **A failed broker.** With one replica it takes the only copy of every event it held. With more replicas, `all` loses none, `1` loses the events the followers had not copied, and `0` loses the producer's latest events. The lost events spend the error budget.
- **Retention.** A consumer further behind than the window loses the oldest events, and the work they carried never happens. A longer window costs nothing but time here, where a real product bills the disk.
- **Ordering.** Round robin with more than one partition loses order, so a superseded write can land after the one that replaced it. The value a consumer writes is the older one, and the game counts it as a stale result.
- **Partitions.** One partition means one consumer, so a slow consumer holds everyone behind it. More partitions read faster and split the order more finely.
- **A hot key.** One key's traffic fills one partition, whatever the partition count, and that partition sets the whole stream's speed. Incident [Hot key](https://wiki.playquorum.dev/incidents/hot-key.md) shows it.
- **No Worker linked.** Nothing consumes the stream, and every task that reaches it is stranded.

### [Instance failure](https://wiki.playquorum.dev/incidents/instance.md)

- **Signs:** The Event Log's card goes grey, its Inspector reads "The host under this VM died. It still bills.", and the requests sent to it are stranded.
- **Do now:** Press **Replace** in its Inspector and pay the price. It comes back cold.
- **Prevent:** Run a second copy: [Load balancing](https://wiki.playquorum.dev/research/load-balancing.md) puts the copies behind a Load Balancer, and [Health checks](https://wiki.playquorum.dev/research/health-checks.md) skip the dead host.

### [Zone outage](https://wiki.playquorum.dev/incidents/zone.md)

- **Signs:** Every Node in one zone goes grey at once.
- **Do now:** Nothing brings the zone back sooner. **Fail over** promotes a **Standby**, and this Component keeps none, so it ends on its own after at most 120 s.
- **Prevent:** Research [Multiple zones](https://wiki.playquorum.dev/research/multi-az.md), then put copies in different zones with the Zone setting.

### [Region outage](https://wiki.playquorum.dev/incidents/region.md)

- **Signs:** Every Node in one region goes grey at once, and that region's board goes grey with them.
- **Do now:** Nothing speeds it up. **Fail over** moves only Nodes that hold no data, so this one stays down until the incident ends: 180 sim seconds in the catalog, 60 once the error budget counts. A store spread across regions needs nothing from you.
- **Prevent:** Research [Multiple regions](https://wiki.playquorum.dev/research/multi-region.md) and [GeoDNS](https://wiki.playquorum.dev/research/geo-dns.md), then run a copy of it in a second region.

## Tips

- Watch lag, not latency.
- Key the stream on the entity whose order matters, such as an order or an account.
- Set more partitions than you have consumers.
- Use `acks=all` with three replicas for money. The cheapest setting is one replica with no acknowledgement at all.

## Rules

- Each further copy costs 25% more than the one before.
- Selling it refunds 50% of what its last copy cost.
- It has 6 sizes. Each step doubles its capacity and its Burn.

## Works with

[Database](https://wiki.playquorum.dev/components/database.md), [Queue](https://wiki.playquorum.dev/components/queue.md), [CDC Connector](https://wiki.playquorum.dev/components/cdc.md)

## Research next

Once you own Event streaming, these Research items build on it.

[Change data capture](https://wiki.playquorum.dev/research/cdc.md)

## Engineering terms

- [At-least-once delivery](https://wiki.playquorum.dev/glossary/at-least-once-delivery.md)
- [CDC (change data capture)](https://wiki.playquorum.dev/glossary/cdc-change-data-capture.md)

## Facts

Event Stream
- **Code:** log
- **Real product:** Apache Kafka
- **Group:** Messaging
- **Unlocked by:** [Event streaming](https://wiki.playquorum.dev/research/event-streaming.md)
- **Era:** [Series B](https://wiki.playquorum.dev/eras/series-b.md)
- **Place:** $900 first copy
- **Burn:** $0.30/s
- **Sizes:** 6
- **Overhead:** 5 ms
- **Takes:** Task
- **Sends:** Task

## See also

- [Event streaming](https://wiki.playquorum.dev/research/event-streaming.md)
- [Components](https://wiki.playquorum.dev/components.md)

Source: https://wiki.playquorum.dev/components/log