# Patterns

> System design in small pieces: build a Design one rung at a time, then look up the pattern behind each rung.

A cheatsheet for designing systems, in two halves that point at each other. The Stack ladder adds one piece at a time and says when to climb. The patterns explain the idea behind each rung, what it costs, and how to set it in the game.

New here? Start at [One box](https://wiki.playquorum.dev/patterns/one-box.md) and climb, or jump to a pattern when a gauge on your Board turns red. Most pages draw a real Design on the game’s own board and explain every setting it uses.

## Foundations

The vocabulary the rest leans on: latency, availability, CAP and the tail.

[Latency vs throughput Latency is how long one request takes, throughput is how many finish per second, and pushing one up often pushes the other the wrong way.](https://wiki.playquorum.dev/patterns/latency-vs-throughput.md)[Availability and the nines Each extra nine of uptime cuts the allowed downtime tenfold, and every hard dependency in a chain pulls the total down.](https://wiki.playquorum.dev/patterns/availability-and-nines.md)[CAP and PACELC When the network splits, a replicated store must refuse some requests or serve stale ones, and even when it is healthy it trades latency for consistency.](https://wiki.playquorum.dev/patterns/cap-and-pacelc.md)[Consistency models A consistency model says which values a read may return after a write, and the stronger ones cost latency and availability.](https://wiki.playquorum.dev/patterns/consistency-models.md)[Little's law The number of requests inside a system equals the arrival rate times the time each one spends there, which sizes every pool, queue and connection limit.](https://wiki.playquorum.dev/patterns/littles-law.md)[Tail latency The slowest few percent of requests set how a service feels, and a request that fans out to many servers meets that tail almost every time.](https://wiki.playquorum.dev/patterns/tail-latency.md)[Fallacies of distributed computing Eight assumptions that hold on one machine and fail across a network, each of which has a standard defense.](https://wiki.playquorum.dev/patterns/fallacies-of-distributed-computing.md)

## Stack ladder

Build a system one rung at a time, from a single box to several regions.

[1 One box Run the code and the data on one machine. It is the cheapest way to launch and the right place to start.](https://wiki.playquorum.dev/patterns/one-box.md)[2 Split the server and the storage Move the data off the App Server into its own Database Node, so code and data stop fighting over one machine and each can grow alone.](https://wiki.playquorum.dev/patterns/split-the-server-and-storage.md)[3 Stateless app servers Keep nothing on the App Server that a second copy would need, so any copy can answer any request and a dead one costs nothing.](https://wiki.playquorum.dev/patterns/stateless-app-servers.md)[4 Scale up first Give the App Server a bigger instance before you add a second one. It is the cheapest change that adds capacity.](https://wiki.playquorum.dev/patterns/scale-up-first.md)[5 Several app servers behind a load balancer Put a Load Balancer in front of identical App Servers so traffic splits across them and capacity grows by adding copies.](https://wiki.playquorum.dev/patterns/several-app-servers-behind-a-load-balancer.md)[6 Drop dead nodes with health checks Have the Load Balancer probe each server and stop sending traffic to one that has failed, so a dead host costs seconds instead of a share of requests.](https://wiki.playquorum.dev/patterns/drop-dead-nodes-with-health-checks.md)[7 Pooler between servers and database Put a Connection Pooler between the App Servers and the Database so many app connections share a few database connections.](https://wiki.playquorum.dev/patterns/pooler-between-servers-and-database.md)[8 DNS in front Put DNS ahead of several Load Balancers so the entry point is not a single machine, and set its TTL to match how fast you need failover.](https://wiki.playquorum.dev/patterns/dns-in-front.md)[9 Static files on object storage and a CDN Serve pages and media from the edge and keep files in Object Storage, so the origin only handles what must be fresh.](https://wiki.playquorum.dev/patterns/static-files-on-object-storage-and-a-cdn.md)[10 Add a cache Put a Cache between the App Server and the Database so repeated reads are answered from memory and the Database keeps its capacity for writes.](https://wiki.playquorum.dev/patterns/add-a-cache.md)[11 Add a read replica Copy the Database to a replica that answers reads, so the primary's capacity goes to writes and read traffic scales out.](https://wiki.playquorum.dev/patterns/add-a-read-replica.md)[12 Route reads and writes apart Send writes to the primary and reads to a replica, and send reads that follow a write back to the primary so users never see their own change missing.](https://wiki.playquorum.dev/patterns/route-reads-and-writes-apart.md)[13 Search beside the database Put full-text search on its own Search Engine, so a busy query stops taking CPU and cache from checkouts and writes.](https://wiki.playquorum.dev/patterns/search-beside-the-database.md)[14 Document and wide-column stores Use a document store for data read and written by key with a changing shape, and a wide-column store when writes outgrow one machine, and accept what each gives up.](https://wiki.playquorum.dev/patterns/document-and-wide-column-stores.md)[15 Split the database by feature Give each feature its own App Server and Database, so one feature's load and failures stay in its own lane.](https://wiki.playquorum.dev/patterns/split-the-database-by-feature.md)[16 Slow work to a queue and workers Hand work the user does not wait for to a Queue, and let Workers drain it at their own pace.](https://wiki.playquorum.dev/patterns/slow-work-to-a-queue-and-workers.md)[17 Message platform for fan-out Send each notification through a Queue and Workers to a managed provider, so a slow or failing send never holds a request.](https://wiki.playquorum.dev/patterns/message-platform-for-fan-out.md)[18 Functions for spiky jobs Run bursty or rare work on a Serverless Function that bills per use and scales from zero, and keep steady load on boxes.](https://wiki.playquorum.dev/patterns/functions-for-spiky-jobs.md)[19 Autoscale the compute tier Link App Servers to an Autoscaler so the number of copies follows the load, inside a floor and a cap you choose.](https://wiki.playquorum.dev/patterns/autoscale-the-compute-tier.md)[20 Stream changes with CDC Read the Database's write log and publish each change to a stream, so search and other copies follow the data without a second write.](https://wiki.playquorum.dev/patterns/stream-changes-with-cdc.md)[21 Reports off the primary, in a warehouse Send report scans to a columnar Data Warehouse, so analytics stops evicting the buffer cache that checkouts depend on.](https://wiki.playquorum.dev/patterns/reports-off-the-primary-warehouse.md)[22 WAF and gateway at the front Put a WAF and an API Gateway in front of the App Servers, so bots and bursts are stopped before they cost you capacity.](https://wiki.playquorum.dev/patterns/waf-and-gateway-at-the-front.md)[23 Auth and TLS at the gateway Check each token once at the API Gateway, hand password hashing to an Auth Provider, and let a certificate manager renew TLS, so the front door stops costing the App Server and stops expiring.](https://wiki.playquorum.dev/patterns/auth-and-tls-at-the-gateway.md)[24 Feature flags and CI for safe deploys Ship through a pipeline that sends a canary first, and put new Features behind flags, so a bad deploy breaks a slice of users instead of all of them.](https://wiki.playquorum.dev/patterns/feature-flags-and-ci-for-safe-deploys.md)[25 A WebSockets tier for realtime Hold realtime connections open on a Realtime Hub that runs a runtime built for them, and route only realtime traffic to it, so chat and live updates stop eating App Server workers.](https://wiki.playquorum.dev/patterns/websockets-tier-for-realtime.md)[26 Shard the database Split the Database into shards by a key, so writes divide across several primaries once one box at its largest size is full.](https://wiki.playquorum.dev/patterns/shard-the-database.md)[27 Distributed SQL and leaderless stores Replace hand-made shards with a store that replicates by itself, either leaderless with tunable quorums or SQL on Raft, and pick by whether you need every read correct.](https://wiki.playquorum.dev/patterns/distributed-sql-and-leaderless-stores.md)[28 Logs, metrics and traces Tap the Nodes into Analytics and research the visibility track, so you can see error causes, history and the slow hop of each request instead of guessing.](https://wiki.playquorum.dev/patterns/logs-metrics-and-traces.md)[29 Split into services with a mesh Break one codebase into Services on a Kubernetes Cluster and put a Service Mesh beside them, so each deploys alone and every call gets retries, a breaker and metrics.](https://wiki.playquorum.dev/patterns/split-into-services-with-a-mesh.md)[30 Multi-region with GeoDNS Open a second region, route each region's Users to it with GeoDNS, and keep reads local, so far Users stop paying the ocean twice on every request.](https://wiki.playquorum.dev/patterns/multi-region-with-geodns.md)[31 Compute at the edge Run small handlers on Edge Functions next to the user and keep heavy work on the App Server, so cheap requests skip the trip to the origin.](https://wiki.playquorum.dev/patterns/compute-at-the-edge.md)[32 Disaster recovery and a failover region Back up the primary, keep a standby for zone loss and a store spread across regions for region loss, and prove it with a drill.](https://wiki.playquorum.dev/patterns/disaster-recovery-and-failover-region.md)[33 An LLM behind a gateway on GPUs Serve AI prompts from your own GPU Nodes beside a hosted LLM API, and tune model, precision and batch size, because a GPU is cheaper per prompt only while it stays busy.](https://wiki.playquorum.dev/patterns/llm-behind-a-gateway-on-gpus.md)[34 Vector search and embeddings Keep embeddings of your documents in a Vector Index and look up the nearest ones before each prompt, so answers use your own data instead of only the model's.](https://wiki.playquorum.dev/patterns/vector-search-and-embeddings.md)

## Scaling reads and compute

Caches, replicas, load balancers and autoscaling.

[Cache-aside Keep hot reads in memory and let the app fill the cache on a miss, so the database serves each popular row once instead of once per request.](https://wiki.playquorum.dev/patterns/cache-aside.md)[Write-through and write-behind Let the cache take writes too, either passing each one straight on to the database or holding it briefly, so reads never see an old value.](https://wiki.playquorum.dev/patterns/write-through-and-write-behind.md)[Cache stampede protection Stop many readers from rebuilding the same expired cache key at once, so one popular key cannot knock the database over.](https://wiki.playquorum.dev/patterns/cache-stampede-protection.md)[Cache warming and tiered caching Fill a cache before it takes traffic, and put a second cache between the first and the origin, so a cold or distant cache does not send every read to the source.](https://wiki.playquorum.dev/patterns/cache-warming-and-tiered-caching.md)[Read replicas Copy the database to followers that serve reads, so read traffic scales out while one primary takes the writes.](https://wiki.playquorum.dev/patterns/read-replicas.md)[CDN push and pull Put copies of your content on servers near users, either filling them on demand from your origin (pull) or uploading them yourself (push).](https://wiki.playquorum.dev/patterns/cdn-push-and-pull.md)[Vertical and horizontal scaling Make one machine bigger, or run more machines, and know which one to reach for first.](https://wiki.playquorum.dev/patterns/vertical-and-horizontal-scaling.md)[Autoscaling, reactive and scheduled Add copies of a service when load rises and remove them when it falls, either after the load shows up or ahead of a peak you can predict.](https://wiki.playquorum.dev/patterns/autoscaling-reactive-and-scheduled.md)[Connection pooling Put a pooler between the app and the database so many app connections share a few database connections.](https://wiki.playquorum.dev/patterns/connection-pooling.md)[Load balancing algorithms Pick how a load balancer chooses among servers, because the right rule depends on whether servers differ and whether requests do.](https://wiki.playquorum.dev/patterns/load-balancing-algorithms.md)[Layer 4 and layer 7 load balancing Choose between a balancer that forwards TCP connections quickly and one that reads each HTTP request so it can route by content.](https://wiki.playquorum.dev/patterns/layer-4-and-layer-7-load-balancing.md)[Fan-out on write and on read Build a user's feed either when someone posts, by copying the post to every follower, or when the user reads, by gathering posts from everyone they follow.](https://wiki.playquorum.dev/patterns/fan-out-on-write-and-on-read.md)[Materialized views and denormalization Store the answer to an expensive query, or keep a joined copy of the data, so reads stop doing the same work again and again.](https://wiki.playquorum.dev/patterns/materialized-views-and-denormalization.md)[Precomputation Do expensive work before anyone asks for it, so a request reads a ready answer instead of building one.](https://wiki.playquorum.dev/patterns/precomputation.md)

## Scaling data

Sharding, replication, quorums and the stores that fit each shape.

[Sharding Split one dataset across several databases by a key, so writes and storage scale past what one machine holds.](https://wiki.playquorum.dev/patterns/sharding.md)[Consistent hashing Place keys on a ring of servers, so adding or removing one server moves only its share of the keys instead of nearly all of them.](https://wiki.playquorum.dev/patterns/consistent-hashing.md)[Hot keys and hot shards When one key or one shard takes a large share of the traffic, adding servers does not help, so spread that key or change the shard key.](https://wiki.playquorum.dev/patterns/hot-keys-and-hot-shards.md)[Federation Split the data by feature into separate databases, so each service owns its tables and one busy feature cannot slow the rest.](https://wiki.playquorum.dev/patterns/federation.md)[Replication strategies Choose how copies of the data stay in step, trading write latency and lost writes against stale reads and complexity.](https://wiki.playquorum.dev/patterns/replication-strategies.md)[Quorum reads and writes Keep several copies of each key and let a read and a write each ask enough of them to overlap, so reads see the latest write without a leader.](https://wiki.playquorum.dev/patterns/quorum-reads-and-writes.md)[Consensus and leader election Make a group of servers agree on one order of writes through a majority vote, so the data stays correct while a minority fails.](https://wiki.playquorum.dev/patterns/consensus-and-leader-election.md)[Read your writes After a user writes, send that user's reads to the primary for a while, so they never see data older than their own change.](https://wiki.playquorum.dev/patterns/read-your-writes.md)[Full-text search and the inverted index Index every word to the documents that hold it in a separate search engine, so a query reads a few short lists instead of scanning every row.](https://wiki.playquorum.dev/patterns/full-text-search-and-inverted-index.md)[OLTP and OLAP split Run reports on a separate columnar warehouse, so large scans stop evicting the cache and slowing the database that serves users.](https://wiki.playquorum.dev/patterns/oltp-and-olap-split.md)[Object storage and lifecycle rules Keep files in object storage and let lifecycle rules move old objects to cheaper classes, so storage cost stops growing in step with every upload.](https://wiki.playquorum.dev/patterns/object-storage-and-lifecycle-rules.md)

## Messaging and async

Queues, streams and the guarantees between them.

[Queue-based load leveling Put a queue between a bursty producer and a steady consumer, so a spike becomes a short wait instead of a failure.](https://wiki.playquorum.dev/patterns/queue-load-leveling.md)[Competing consumers Let several workers read from one queue, so each task goes to exactly one of them and throughput grows by adding workers.](https://wiki.playquorum.dev/patterns/competing-consumers.md)[Publish and subscribe Publish each event once to a topic and let every interested service read it, so producers stop knowing who consumes their events.](https://wiki.playquorum.dev/patterns/publish-subscribe.md)[Dead-letter queue Move a message that keeps failing to a separate queue after a few attempts, so one bad message cannot block the workers.](https://wiki.playquorum.dev/patterns/dead-letter-queue.md)[Backpressure Make a slow consumer push back on its producers, so a queue stays bounded instead of growing until memory or latency runs out.](https://wiki.playquorum.dev/patterns/backpressure.md)[Idempotency keys Tag each write with a unique key the server remembers, so a retried request does the work once.](https://wiki.playquorum.dev/patterns/idempotency-keys.md)[Transactional outbox Write the event in the same database transaction as the change, then publish it from there, so the database and the broker never disagree.](https://wiki.playquorum.dev/patterns/transactional-outbox.md)[Change data capture Read the database's change log and publish each change as an event, so search, warehouse, and cache stay in step without the app writing twice.](https://wiki.playquorum.dev/patterns/change-data-capture.md)[Claim check Put a large payload in object storage and send only a reference through the queue, so messages stay small and fast.](https://wiki.playquorum.dev/patterns/claim-check.md)

## Resilience

Timeouts, retries, breakers and the other ways to fail small.

[Timeouts Give every network call a deadline, because a call that waits forever holds a thread, a connection and a user hostage.](https://wiki.playquorum.dev/patterns/timeouts.md)[Retry with backoff and jitter Retry a failed call after a growing, randomized wait, so a brief fault heals instead of turning into a retry storm.](https://wiki.playquorum.dev/patterns/retry-with-backoff.md)[Circuit breaker Stop calling a dependency that keeps failing, answer fast with a fallback, and probe now and then to see whether it has recovered.](https://wiki.playquorum.dev/patterns/circuit-breaker.md)[Bulkhead Give each dependency or customer its own pool of capacity, so one failing part cannot drain the resources every other part needs.](https://wiki.playquorum.dev/patterns/bulkhead.md)[Load shedding When demand passes capacity, refuse the cheapest work early so the work you keep still finishes on time.](https://wiki.playquorum.dev/patterns/load-shedding.md)[Rate limiting Cap how many requests a client or a service may send per second, so one noisy caller cannot use up capacity that everyone shares.](https://wiki.playquorum.dev/patterns/rate-limiting.md)[Hedged requests Send a second copy of a slow call and keep the first answer, so a rare slow server stops setting your tail latency.](https://wiki.playquorum.dev/patterns/hedged-requests.md)[Graceful degradation When a system cannot do everything, switch off the costly features first, so the core of the product keeps working.](https://wiki.playquorum.dev/patterns/graceful-degradation.md)[Health checks and failover Probe each server and stop sending it traffic when it stops answering, then move to a standby when a whole zone goes.](https://wiki.playquorum.dev/patterns/health-checks-and-failover.md)[Chaos engineering Break your own system on a schedule and under watch, so you find the weak link before an outage does.](https://wiki.playquorum.dev/patterns/chaos-engineering.md)

## Deploys and change

Ship and migrate without taking the system down.

[Expand and contract Change a database schema in small steps that old and new code both accept, so a deploy never needs downtime or a lockstep release.](https://wiki.playquorum.dev/patterns/expand-and-contract.md)[Blue-green and canary deploys Ship a new version beside the old one, or to a few users first, so a bad release reaches few people and rolls back fast.](https://wiki.playquorum.dev/patterns/blue-green-and-canary.md)[Feature flags Put a new feature behind a switch you can turn up or down at runtime, so deploying code and releasing it are separate steps.](https://wiki.playquorum.dev/patterns/feature-flags.md)[Strangler pattern Replace an old system piece by piece behind one front door, so the new one takes over without a big-bang rewrite.](https://wiki.playquorum.dev/patterns/strangler-pattern.md)[Infrastructure as code and GitOps Describe your infrastructure in files and apply it from version control, so every environment is the same and every change has a history.](https://wiki.playquorum.dev/patterns/infrastructure-as-code-and-gitops.md)[Online migrations Change a live database without locking it, so writes keep flowing while the schema or the data moves.](https://wiki.playquorum.dev/patterns/online-migrations.md)

## Operate and observe

See the system, set targets, and plan for the day it breaks.

[Golden signals, RED and USE Watch four things on every service, latency, traffic, errors and saturation, and check three more on every resource, so a dashboard answers "what is broken" before anyone opens a log.](https://wiki.playquorum.dev/patterns/golden-signals-red-and-use.md)[SLOs and error budgets Pick a reliability target users would notice, treat the failures it allows as a budget, and let the remaining budget decide whether to ship or to harden.](https://wiki.playquorum.dev/patterns/slos-and-error-budgets.md)[Percentiles, not averages Report p50, p95 and p99 instead of the mean, because a mean averages away the slow requests that real users complain about.](https://wiki.playquorum.dev/patterns/percentiles-not-averages.md)[Distributed tracing and tail sampling Follow one request across every service as a trace, and decide after it finishes which traces to keep, so the slow and failed ones survive sampling.](https://wiki.playquorum.dev/patterns/distributed-tracing-and-tail-sampling.md)[Runbooks and on-call Put one person on call at a time, write down the fix for each failure the team has already handled, and automate the ones that repeat.](https://wiki.playquorum.dev/patterns/runbooks-and-on-call.md)[Disaster recovery tiers Choose how much standby capacity to keep for a lost region, from backups alone to a second site that serves traffic all the time, and pay for the speed you need.](https://wiki.playquorum.dev/patterns/disaster-recovery-tiers.md)[RPO and RTO Set two targets for every disaster, how much data you may lose and how long you may be down, and let them pick the backup and standby design.](https://wiki.playquorum.dev/patterns/rpo-and-rto.md)

## Security and edge

The front door, encryption, secrets and where data may live.

[API gateway patterns Put one front door before your services, so rate limits, authentication and request merging happen once instead of in every service.](https://wiki.playquorum.dev/patterns/api-gateway-patterns.md)[WAF and bot management Filter hostile traffic at the edge with a firewall that blocks known attack patterns, and challenge suspicious clients instead of blocking them so real users still get through.](https://wiki.playquorum.dev/patterns/waf-and-bot-management.md)[Encryption in transit and at rest Encrypt data on the wire with TLS and on disk with keys you control, and automate certificate renewal so an expiry date never takes the site down.](https://wiki.playquorum.dev/patterns/encryption-in-transit-and-at-rest.md)[Secrets management Keep keys, passwords and tokens in a vault that hands them out on demand, so none sits in code, in an image or on a phishing page's wish list.](https://wiki.playquorum.dev/patterns/secrets-management.md)[Dependency scanning Check every build's libraries against known-vulnerability lists, so a flawed or malicious package fails the pipeline before it reaches production.](https://wiki.playquorum.dev/patterns/dependency-scanning.md)[Data residency Keep a region's personal data on servers in that region, and route its users there, so local law and customer contracts are met without a separate product.](https://wiki.playquorum.dev/patterns/data-residency.md)

## Cost

Pay for what you use and know what each user costs.

[Rightsizing, spot and reserved Match each machine to the load it carries, then buy the steady part on a commitment and the interruptible part at a deep discount.](https://wiki.playquorum.dev/patterns/rightsizing-spot-and-reserved.md)[Egress optimization Cut the bill for bytes that leave your cloud by compressing responses, serving them from a CDN, and keeping chatty traffic inside one zone and region.](https://wiki.playquorum.dev/patterns/egress-optimization.md)[Unit economics and cost allocation Divide the bill by what it buys, so each request kind, service and team carries its own cost, and "the cloud bill went up" becomes a specific answer.](https://wiki.playquorum.dev/patterns/unit-economics-and-cost-allocation.md)

## Outside the game

Patterns Quorum cannot build, with references worth reading.

[Saga A saga runs a business transaction as a chain of local transactions across services and undoes finished steps with compensating actions when a later step fails.](https://wiki.playquorum.dev/patterns/saga.md)[CQRS CQRS uses one model to change data and a different one to read it, so each side can be shaped and scaled for its own job.](https://wiki.playquorum.dev/patterns/cqrs.md)[Event sourcing Event sourcing stores every change as an event in an append-only log and builds current state by replaying it.](https://wiki.playquorum.dev/patterns/event-sourcing.md)[Sidecar A sidecar runs next to the main application in its own process or container and handles shared chores like logging, proxying and security.](https://wiki.playquorum.dev/patterns/sidecar.md)[Anti-corruption layer An anti-corruption layer translates between your model and a legacy or foreign system's, so the old design does not leak into the new one.](https://wiki.playquorum.dev/patterns/anti-corruption-layer.md)[Backends for frontends Backends for frontends gives each kind of client, such as web, iOS and Android, its own small backend instead of one shared API.](https://wiki.playquorum.dev/patterns/backends-for-frontends.md)[API composition API composition answers a query that spans several services by calling each one and joining the results in memory.](https://wiki.playquorum.dev/patterns/api-composition.md)[Two-phase commit Two-phase commit makes several databases commit or abort one transaction together, by voting first and committing second, at the price of blocking when the coordinator fails.](https://wiki.playquorum.dev/patterns/two-phase-commit.md)[CRDTs A conflict-free replicated data type lets every replica accept writes on its own and still merge to the same state without coordination.](https://wiki.playquorum.dev/patterns/crdts.md)[Sticky sessions Sticky sessions make a load balancer send a client back to the server it used first, so that server's memory can hold the client's state.](https://wiki.playquorum.dev/patterns/sticky-sessions.md)[Long polling and server-sent events Long polling and server-sent events let a server push updates to a browser over plain HTTP, without the two-way connection of a WebSocket.](https://wiki.playquorum.dev/patterns/long-polling-and-sse.md)

Source: https://wiki.playquorum.dev/patterns