Fan-out on write and on read

Fan-out on write does the work at post time, and fan-out on read does it at view time.

Use it when#

Reads of the feed far outnumber posts, so the cost belongs on the write.
A feed must load fast, because it is the first screen.
Accounts have a bounded number of followers, or you can treat the huge ones separately.

How it works#

On read (pull), the feed request asks for the latest posts of each followed account and merges them. Posting is cheap and reading is slow. On write (push), a post is copied into each follower's stored timeline, so reading is one lookup. If a user has 200 followers, one post makes 200 small writes.

The problem is the account with 50 million followers, where one post means 50 million writes. Twitter's published design pushes for most accounts and pulls the posts of the most followed ones at read time. This hybrid is the common answer.

What it costs#

Push multiplies writes, stores a timeline per user, and wastes work on followers who never log in. Pull makes the busiest request the most expensive and slow at the tail. The hybrid needs a rule for who counts as large, and the merge adds complexity.

In Quorum#

Fan-out is a setting on the App Server: pull or push. Push writes timelines through the data tier. Researching Hybrid fan-out turns push into the hybrid and lets a Cache split hot keys. Feeds start in a late Era, so both Designs run there with the other kinds' backends linked.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 400 Users the Simulation solves this Design to Apdex 0.89, p99 461 ms, 0.0% errors and $6.40/s Burn.

Where	Setting	Value	Why
App Server	Fan-out	pull	A feed request reads the latest posts of each account the user follows, so one read becomes many queries.
App Server	Instance size	small	Small gives the app two vCPUs, enough to run code while the data tier does its own work.
Cache	Instance size	small	Memory decides how much of the hot set fits. A Cache too small for it misses even on popular rows.
Database	Instance size	small	Small gives the Database two vCPUs, so the figures below reflect the Cache and not a starved Database.

Research this Design needs: Caching, Checkout, Hosted models, Notifications.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 400 Users the Simulation solves this Design to Apdex 0.94, p99 408 ms, 0.0% errors and $6.40/s Burn.

Where	Setting	Value	Why
App Server	Fan-out	push	Each post is written into followers' timelines, so a feed request reads one list. Writes cost more instead. With Hybrid fan-out researched, posts from the most followed accounts are pulled at read time and not pushed.
App Server	Instance size	small	Small gives the app two vCPUs, enough to run code while the data tier does its own work.
Cache	Split hot keys	On	A famous account's timeline is spread over several shards, so no one shard takes every read of it.
Cache	Instance size	small	Memory decides how much of the hot set fits. A Cache too small for it misses even on popular rows.
Database	Instance size	small	Small gives the Database two vCPUs, so the figures below reflect the Cache and not a starved Database.

Research this Design needs: Caching, Checkout, Hosted models, Notifications, Hybrid fan-out.

The Hot key incident is the celebrity problem. The glossary defines fan-out and fan-out on write.