Patterns

Latency is how long one request takes, throughput is how many finish per second, and pushing one up often pushes the other the wrong way.

Each extra nine of uptime cuts the allowed downtime tenfold, and every hard dependency in a chain pulls the total down.

When the network splits, a replicated store must refuse some requests or serve stale ones, and even when it is healthy it trades latency for consistency.

A consistency model says which values a read may return after a write, and the stronger ones cost latency and availability.

The number of requests inside a system equals the arrival rate times the time each one spends there, which sizes every pool, queue and connection limit.

The slowest few percent of requests set how a service feels, and a request that fans out to many servers meets that tail almost every time.

Eight assumptions that hold on one machine and fail across a network, each of which has a standard defense.

Run the code and the data on one machine. It is the cheapest way to launch and the right place to start.

Move the data off the App Server into its own Database Node, so code and data stop fighting over one machine and each can grow alone.

Keep nothing on the App Server that a second copy would need, so any copy can answer any request and a dead one costs nothing.

Give the App Server a bigger instance before you add a second one. It is the cheapest change that adds capacity.

Put a Load Balancer in front of identical App Servers so traffic splits across them and capacity grows by adding copies.

Have the Load Balancer probe each server and stop sending traffic to one that has failed, so a dead host costs seconds instead of a share of requests.

Put a Connection Pooler between the App Servers and the Database so many app connections share a few database connections.

Put DNS ahead of several Load Balancers so the entry point is not a single machine, and set its TTL to match how fast you need failover.

Serve pages and media from the edge and keep files in Object Storage, so the origin only handles what must be fresh.

Put a Cache between the App Server and the Database so repeated reads are answered from memory and the Database keeps its capacity for writes.

Copy the Database to a replica that answers reads, so the primary's capacity goes to writes and read traffic scales out.

Send writes to the primary and reads to a replica, and send reads that follow a write back to the primary so users never see their own change missing.

Put full-text search on its own Search Engine, so a busy query stops taking CPU and cache from checkouts and writes.

Use a document store for data read and written by key with a changing shape, and a wide-column store when writes outgrow one machine, and accept what each gives up.

Give each feature its own App Server and Database, so one feature's load and failures stay in its own lane.

Hand work the user does not wait for to a Queue, and let Workers drain it at their own pace.

Send each notification through a Queue and Workers to a managed provider, so a slow or failing send never holds a request.

Run bursty or rare work on a Serverless Function that bills per use and scales from zero, and keep steady load on boxes.

Link App Servers to an Autoscaler so the number of copies follows the load, inside a floor and a cap you choose.

Read the Database's write log and publish each change to a stream, so search and other copies follow the data without a second write.

Send report scans to a columnar Data Warehouse, so analytics stops evicting the buffer cache that checkouts depend on.

Put a WAF and an API Gateway in front of the App Servers, so bots and bursts are stopped before they cost you capacity.

Check each token once at the API Gateway, hand password hashing to an Auth Provider, and let a certificate manager renew TLS, so the front door stops costing the App Server and stops expiring.

Ship through a pipeline that sends a canary first, and put new Features behind flags, so a bad deploy breaks a slice of users instead of all of them.

Hold realtime connections open on a Realtime Hub that runs a runtime built for them, and route only realtime traffic to it, so chat and live updates stop eating App Server workers.

Split the Database into shards by a key, so writes divide across several primaries once one box at its largest size is full.

Replace hand-made shards with a store that replicates by itself, either leaderless with tunable quorums or SQL on Raft, and pick by whether you need every read correct.

Tap the Nodes into Analytics and research the visibility track, so you can see error causes, history and the slow hop of each request instead of guessing.

Break one codebase into Services on a Kubernetes Cluster and put a Service Mesh beside them, so each deploys alone and every call gets retries, a breaker and metrics.

Open a second region, route each region's Users to it with GeoDNS, and keep reads local, so far Users stop paying the ocean twice on every request.

Run small handlers on Edge Functions next to the user and keep heavy work on the App Server, so cheap requests skip the trip to the origin.

Back up the primary, keep a standby for zone loss and a store spread across regions for region loss, and prove it with a drill.

Serve AI prompts from your own GPU Nodes beside a hosted LLM API, and tune model, precision and batch size, because a GPU is cheaper per prompt only while it stays busy.

Keep embeddings of your documents in a Vector Index and look up the nearest ones before each prompt, so answers use your own data instead of only the model's.

Keep hot reads in memory and let the app fill the cache on a miss, so the database serves each popular row once instead of once per request.

Let the cache take writes too, either passing each one straight on to the database or holding it briefly, so reads never see an old value.

Stop many readers from rebuilding the same expired cache key at once, so one popular key cannot knock the database over.

Fill a cache before it takes traffic, and put a second cache between the first and the origin, so a cold or distant cache does not send every read to the source.

Copy the database to followers that serve reads, so read traffic scales out while one primary takes the writes.

Put copies of your content on servers near users, either filling them on demand from your origin (pull) or uploading them yourself (push).

Make one machine bigger, or run more machines, and know which one to reach for first.

Add copies of a service when load rises and remove them when it falls, either after the load shows up or ahead of a peak you can predict.

Put a pooler between the app and the database so many app connections share a few database connections.

Pick how a load balancer chooses among servers, because the right rule depends on whether servers differ and whether requests do.

Choose between a balancer that forwards TCP connections quickly and one that reads each HTTP request so it can route by content.

Build a user's feed either when someone posts, by copying the post to every follower, or when the user reads, by gathering posts from everyone they follow.

Store the answer to an expensive query, or keep a joined copy of the data, so reads stop doing the same work again and again.

Do expensive work before anyone asks for it, so a request reads a ready answer instead of building one.

Split one dataset across several databases by a key, so writes and storage scale past what one machine holds.

Place keys on a ring of servers, so adding or removing one server moves only its share of the keys instead of nearly all of them.

When one key or one shard takes a large share of the traffic, adding servers does not help, so spread that key or change the shard key.

Split the data by feature into separate databases, so each service owns its tables and one busy feature cannot slow the rest.

Choose how copies of the data stay in step, trading write latency and lost writes against stale reads and complexity.

Keep several copies of each key and let a read and a write each ask enough of them to overlap, so reads see the latest write without a leader.

Make a group of servers agree on one order of writes through a majority vote, so the data stays correct while a minority fails.

After a user writes, send that user's reads to the primary for a while, so they never see data older than their own change.

Index every word to the documents that hold it in a separate search engine, so a query reads a few short lists instead of scanning every row.

Run reports on a separate columnar warehouse, so large scans stop evicting the cache and slowing the database that serves users.

Keep files in object storage and let lifecycle rules move old objects to cheaper classes, so storage cost stops growing in step with every upload.

Put a queue between a bursty producer and a steady consumer, so a spike becomes a short wait instead of a failure.

Let several workers read from one queue, so each task goes to exactly one of them and throughput grows by adding workers.

Publish each event once to a topic and let every interested service read it, so producers stop knowing who consumes their events.

Move a message that keeps failing to a separate queue after a few attempts, so one bad message cannot block the workers.

Make a slow consumer push back on its producers, so a queue stays bounded instead of growing until memory or latency runs out.

Tag each write with a unique key the server remembers, so a retried request does the work once.

Write the event in the same database transaction as the change, then publish it from there, so the database and the broker never disagree.

Read the database's change log and publish each change as an event, so search, warehouse, and cache stay in step without the app writing twice.

Put a large payload in object storage and send only a reference through the queue, so messages stay small and fast.

Give every network call a deadline, because a call that waits forever holds a thread, a connection and a user hostage.

Retry a failed call after a growing, randomized wait, so a brief fault heals instead of turning into a retry storm.

Stop calling a dependency that keeps failing, answer fast with a fallback, and probe now and then to see whether it has recovered.

Give each dependency or customer its own pool of capacity, so one failing part cannot drain the resources every other part needs.

When demand passes capacity, refuse the cheapest work early so the work you keep still finishes on time.

Cap how many requests a client or a service may send per second, so one noisy caller cannot use up capacity that everyone shares.

Send a second copy of a slow call and keep the first answer, so a rare slow server stops setting your tail latency.

When a system cannot do everything, switch off the costly features first, so the core of the product keeps working.

Probe each server and stop sending it traffic when it stops answering, then move to a standby when a whole zone goes.

Break your own system on a schedule and under watch, so you find the weak link before an outage does.

Change a database schema in small steps that old and new code both accept, so a deploy never needs downtime or a lockstep release.

Ship a new version beside the old one, or to a few users first, so a bad release reaches few people and rolls back fast.

Put a new feature behind a switch you can turn up or down at runtime, so deploying code and releasing it are separate steps.

Replace an old system piece by piece behind one front door, so the new one takes over without a big-bang rewrite.

Describe your infrastructure in files and apply it from version control, so every environment is the same and every change has a history.

Change a live database without locking it, so writes keep flowing while the schema or the data moves.

Watch four things on every service, latency, traffic, errors and saturation, and check three more on every resource, so a dashboard answers "what is broken" before anyone opens a log.

Pick a reliability target users would notice, treat the failures it allows as a budget, and let the remaining budget decide whether to ship or to harden.

Report p50, p95 and p99 instead of the mean, because a mean averages away the slow requests that real users complain about.

Follow one request across every service as a trace, and decide after it finishes which traces to keep, so the slow and failed ones survive sampling.

Put one person on call at a time, write down the fix for each failure the team has already handled, and automate the ones that repeat.

Choose how much standby capacity to keep for a lost region, from backups alone to a second site that serves traffic all the time, and pay for the speed you need.

Set two targets for every disaster, how much data you may lose and how long you may be down, and let them pick the backup and standby design.

Put one front door before your services, so rate limits, authentication and request merging happen once instead of in every service.

Filter hostile traffic at the edge with a firewall that blocks known attack patterns, and challenge suspicious clients instead of blocking them so real users still get through.

Encrypt data on the wire with TLS and on disk with keys you control, and automate certificate renewal so an expiry date never takes the site down.

Keep keys, passwords and tokens in a vault that hands them out on demand, so none sits in code, in an image or on a phishing page's wish list.

Check every build's libraries against known-vulnerability lists, so a flawed or malicious package fails the pipeline before it reaches production.

Keep a region's personal data on servers in that region, and route its users there, so local law and customer contracts are met without a separate product.

Match each machine to the load it carries, then buy the steady part on a commitment and the interruptible part at a deep discount.

Cut the bill for bytes that leave your cloud by compressing responses, serving them from a CDN, and keeping chatty traffic inside one zone and region.

Divide the bill by what it buys, so each request kind, service and team carries its own cost, and "the cloud bill went up" becomes a specific answer.

A saga runs a business transaction as a chain of local transactions across services and undoes finished steps with compensating actions when a later step fails.

CQRS uses one model to change data and a different one to read it, so each side can be shaped and scaled for its own job.

Event sourcing stores every change as an event in an append-only log and builds current state by replaying it.

A sidecar runs next to the main application in its own process or container and handles shared chores like logging, proxying and security.

An anti-corruption layer translates between your model and a legacy or foreign system's, so the old design does not leak into the new one.

Backends for frontends gives each kind of client, such as web, iOS and Android, its own small backend instead of one shared API.

API composition answers a query that spans several services by calling each one and joining the results in memory.

Two-phase commit makes several databases commit or abort one transaction together, by voting first and committing second, at the price of blocking when the coordinator fails.

A conflict-free replicated data type lets every replica accept writes on its own and still merge to the same state without coordination.

Sticky sessions make a load balancer send a client back to the server it used first, so that server's memory can hold the client's state.

Long polling and server-sent events let a server push updates to a browser over plain HTTP, without the two-way connection of a WebSocket.