# System design concepts

> The concepts behind most system design questions, grouped by the problem each one solves, with a page for every one and a way to try it in the game.

Most system design questions come down to about fifty ideas, and each one answers a specific problem: too much traffic, data that does not fit on one machine, a dependency that fails, a number that hides the real picture. This page groups them by problem. Every entry links to a page that explains it, and most have a Component or Sandbox in Quorum where you can watch it work on live traffic.

## When one server is not enough

Start by making the app tier something you can copy. [Stateless app servers](https://wiki.playquorum.dev/patterns/stateless-app-servers.md) keep no session in memory, so any copy can answer any request. Then [several app servers behind a load balancer](https://wiki.playquorum.dev/patterns/several-app-servers-behind-a-load-balancer.md) spread the work. The choice between [vertical and horizontal scaling](https://wiki.playquorum.dev/patterns/vertical-and-horizontal-scaling.md) is a cost question, and [scale up first](https://wiki.playquorum.dev/patterns/scale-up-first.md) argues that a bigger box is often the cheaper answer for longer than people expect.

- [Load balancing algorithms](https://wiki.playquorum.dev/patterns/load-balancing-algorithms.md): round robin, least connections, and what each does when one server is slow.
- [Layer 4 and layer 7 load balancing](https://wiki.playquorum.dev/patterns/layer-4-and-layer-7-load-balancing.md): routing by connection or by request contents.
- [Sticky sessions](https://wiki.playquorum.dev/patterns/sticky-sessions.md): pinning a user to one server, and why it makes scaling down harder.
- [Autoscaling the compute tier](https://wiki.playquorum.dev/patterns/autoscale-the-compute-tier.md): adding copies when load rises, and why a slow start makes it arrive late.

## Reading the same data again and again

A cache answers repeated reads from memory so the database does not have to. [Cache-aside](https://wiki.playquorum.dev/patterns/cache-aside.md) is the default, and [write-through and write-behind](https://wiki.playquorum.dev/patterns/write-through-and-write-behind.md) change when a write reaches the database. A hot key expiring can send every reader to the database at once, which [cache stampede protection](https://wiki.playquorum.dev/patterns/cache-stampede-protection.md) prevents.

- [CDN push and pull](https://wiki.playquorum.dev/patterns/cdn-push-and-pull.md): serving files from a server near the reader.
- [Static files on object storage and a CDN](https://wiki.playquorum.dev/patterns/static-files-on-object-storage-and-a-cdn.md): taking images and downloads off the app servers entirely.
- [Cache warming and tiered caching](https://wiki.playquorum.dev/patterns/cache-warming-and-tiered-caching.md): starting warm after a restart, and layering caches.
- [Precomputation](https://wiki.playquorum.dev/patterns/precomputation.md): doing the expensive work before anyone asks.

## When the database is the limit

Reads and writes hit the database differently, so they scale differently. [Read replicas](https://wiki.playquorum.dev/patterns/read-replicas.md) copy the data to servers that only answer reads, and [replication strategies](https://wiki.playquorum.dev/patterns/replication-strategies.md) explain what happens to a write on the way. When writes are the limit, [sharding](https://wiki.playquorum.dev/patterns/sharding.md) splits the data across servers, and [consistent hashing](https://wiki.playquorum.dev/patterns/consistent-hashing.md) decides which server owns a key without moving most keys when the set of servers changes.

- [Hot keys and hot shards](https://wiki.playquorum.dev/patterns/hot-keys-and-hot-shards.md): one key that gets most of the traffic defeats an even split.
- [Connection pooling](https://wiki.playquorum.dev/patterns/connection-pooling.md): a database accepts few connections, and many app servers can use them all.
- [Materialized views and denormalization](https://wiki.playquorum.dev/patterns/materialized-views-and-denormalization.md): storing the answer instead of computing it per read.
- [Document and wide-column stores](https://wiki.playquorum.dev/patterns/document-and-wide-column-stores.md): when the data changes shape or writes outgrow one machine.
- [Full-text search and the inverted index](https://wiki.playquorum.dev/patterns/full-text-search-and-inverted-index.md): why a search engine sits beside the database.
- [OLTP and OLAP split](https://wiki.playquorum.dev/patterns/oltp-and-olap-split.md): keeping reports off the database that takes orders.

## When copies of the data disagree

Once data lives in more than one place, the copies can differ for a while, and a design has to say how long that is acceptable. [CAP and PACELC](https://wiki.playquorum.dev/patterns/cap-and-pacelc.md) state the tradeoff, and [consistency models](https://wiki.playquorum.dev/patterns/consistency-models.md) list the choices from strict to eventual. [Quorum reads and writes](https://wiki.playquorum.dev/patterns/quorum-reads-and-writes.md) let a store pick its point on that line per request.

- [Consensus and leader election](https://wiki.playquorum.dev/patterns/consensus-and-leader-election.md): how a group of servers agrees on one value or one leader.
- [Read your writes](https://wiki.playquorum.dev/patterns/read-your-writes.md): making sure a user sees what they just saved.
- [Two-phase commit](https://wiki.playquorum.dev/patterns/two-phase-commit.md) and the [saga](https://wiki.playquorum.dev/patterns/saga.md): two ways to change data in several places, and what each costs when a step fails.
- [CRDTs](https://wiki.playquorum.dev/patterns/crdts.md): data types that merge without a coordinator.

## Work that does not need to happen now

A queue lets a fast producer hand work to a slower consumer. [Slow work to a queue and workers](https://wiki.playquorum.dev/patterns/slow-work-to-a-queue-and-workers.md) is the basic move, and [queue load leveling](https://wiki.playquorum.dev/patterns/queue-load-leveling.md) shows why it smooths a spike. [Publish-subscribe](https://wiki.playquorum.dev/patterns/publish-subscribe.md) sends one event to many consumers, and [competing consumers](https://wiki.playquorum.dev/patterns/competing-consumers.md) share one queue between many workers.

- [Dead-letter queue](https://wiki.playquorum.dev/patterns/dead-letter-queue.md): where a message goes after it fails too many times.
- [Idempotency keys](https://wiki.playquorum.dev/patterns/idempotency-keys.md): making a retried request safe to repeat.
- [Transactional outbox](https://wiki.playquorum.dev/patterns/transactional-outbox.md): saving a change and announcing it without losing one of the two.
- [Change data capture](https://wiki.playquorum.dev/patterns/change-data-capture.md): turning database changes into events.
- [Fan-out on write and on read](https://wiki.playquorum.dev/patterns/fan-out-on-write-and-on-read.md): doing the work when a post is made or when it is read.
- [Long polling and SSE](https://wiki.playquorum.dev/patterns/long-polling-and-sse.md) and [WebSockets](https://wiki.playquorum.dev/patterns/websockets-tier-for-realtime.md): pushing updates to a client.

## When something fails

A slow dependency is more dangerous than a dead one, because callers wait and pile up. [Timeouts](https://wiki.playquorum.dev/patterns/timeouts.md) bound the wait. [Retry with backoff](https://wiki.playquorum.dev/patterns/retry-with-backoff.md) recovers from short faults, and the [circuit breaker](https://wiki.playquorum.dev/patterns/circuit-breaker.md) stops retries from hammering a service that is down. The [bulkhead](https://wiki.playquorum.dev/patterns/bulkhead.md) keeps one failing dependency from using every thread.

- [Load shedding](https://wiki.playquorum.dev/patterns/load-shedding.md): dropping the least valuable work to keep the rest fast.
- [Rate limiting](https://wiki.playquorum.dev/patterns/rate-limiting.md): capping how much any one client can send.
- [Backpressure](https://wiki.playquorum.dev/patterns/backpressure.md): telling a producer to slow down.
- [Graceful degradation](https://wiki.playquorum.dev/patterns/graceful-degradation.md): turning off a feature instead of failing the whole page.
- [Hedged requests](https://wiki.playquorum.dev/patterns/hedged-requests.md): sending a second request when the first is slow.
- [Health checks and failover](https://wiki.playquorum.dev/patterns/health-checks-and-failover.md): noticing a dead server and sending traffic elsewhere.
- [Chaos engineering](https://wiki.playquorum.dev/patterns/chaos-engineering.md): breaking things on purpose to find out first.

## Measuring a system

You cannot fix what you read wrongly. [Percentiles, not averages](https://wiki.playquorum.dev/patterns/percentiles-not-averages.md) shows how a mean hides the slowest users, and [tail latency](https://wiki.playquorum.dev/patterns/tail-latency.md) explains why the slowest requests dominate once a page makes many calls. [Latency vs throughput](https://wiki.playquorum.dev/patterns/latency-vs-throughput.md) and [Little's law](https://wiki.playquorum.dev/patterns/littles-law.md) connect the two numbers.

- [Availability and nines](https://wiki.playquorum.dev/patterns/availability-and-nines.md): what 99.9 and 99.99 percent mean in minutes of downtime.
- [SLOs and error budgets](https://wiki.playquorum.dev/patterns/slos-and-error-budgets.md): agreeing how much failure is acceptable.
- [Golden signals, RED and USE](https://wiki.playquorum.dev/patterns/golden-signals-red-and-use.md): the few metrics worth watching first.
- [RPO and RTO](https://wiki.playquorum.dev/patterns/rpo-and-rto.md): how much data a disaster may cost and how long recovery may take.

## Common questions

### What are the most important system design concepts to learn first?

Learn load balancing, caching, replication, sharding, and queues first, because almost every interview question uses at least three of them. Add timeouts, retries, and percentiles next, since they explain why systems fail under load and how to notice it.

### How do I practice system design without an interview partner?

Build a system, put traffic through it, and see which part fails first. Quorum does this on a board: you place the Components, the Simulation sends requests through them, and each fix moves the bottleneck somewhere else, so you learn the order in which real systems break. The [first bottleneck tutorial](https://wiki.playquorum.dev/tutorials/first-bottleneck.md) is a short start.

### Is system design only for interviews?

No. The same ideas decide whether a real service survives a launch day or a flash sale, and the [incidents](https://wiki.playquorum.dev/incidents.md) pages show what each one looks like when it goes wrong. Interviews test the same reasoning, with a shorter clock and less data.

Source: https://wiki.playquorum.dev/explanation/system-design-concepts
