ハウツーガイド

Cloudflare のようなコンテンツ配信ネットワークを設計する

システム設計の面接のために CDN を設計します。エッジキャッシュ、アニーキャストと DNS のルーティング、キャッシュ階層、パージ、オリジンシールド、そして Quorum が教えることを扱います。

A CDN is many cache servers in many cities, with a routing layer that sends each user to a nearby one. An edge server answers from its cache when it can, asks a regional cache when it cannot, and only then asks the origin. Routing is done by anycast or DNS, and a purge system removes stale content everywhere.

This is one of the questions in the System Design Primer's list of additional questions.

Requirements#

Functional:

  • Serve static files, images and video from a location near the user.

  • Fetch from the customer's origin on a miss, and cache by rules and TTL.

  • Purge a URL or a tag across the whole network.

Non-functional:

  • Low latency, ideally one short hop from the user.

  • Absorb traffic far above what the origin could take, including attacks.

  • Keep serving when a data center or a link fails.

Estimate the load#

Assume 200 points of presence, 50 Tbps of peak traffic across the network, and an average object of 500 KB.

  • Request rate: 50 Tbps is 6.25 TB a second. Divided by 500 KB, that is about 12.5 million requests a second.

  • Per point of presence: 12.5 million divided by 200 is about 62,000 requests a second on average, and the biggest sites take several times that.

  • Origin load: at a 95% hit rate, 5% of 12.5 million is 625,000 requests a second reaching origins in total. Across a million customers that is small, but one customer's cold object can still draw every edge at once.

  • Storage: a point of presence caching 50 TB of the hottest content needs hundreds of disks, so edges use RAM for the hottest objects and SSD for the rest.

Bandwidth dominates the cost, and the hit rate decides what origins see.

The design#

Routing the user#

Two methods bring a user to an edge. With GeoDNS, the CDN's DNS server answers with the address of a nearby point of presence, chosen by the resolver's location and server health. With anycast, every point of presence announces the same IP address over BGP, and the internet's routing sends each packet to the topologically nearest one. Cloudflare uses anycast, and traffic shifts away from a failed site when its announcement is withdrawn. DNS gives finer control, and anycast reacts faster and also spreads attack traffic.

Edge cache#

The edge terminates TLS, then looks up the object by cache key, usually the URL plus selected headers. A hit returns from memory or SSD. Each server holds a slice of the point of presence's content, picked by hashing the key, so the site caches more distinct objects and not the same hot ones in every box. Eviction is LRU or a variant. The cache key must leave out tracking parameters and include anything that changes the response, or users see each other's pages.

Cache hierarchy and origin shield#

An edge that misses asks an upper-tier cache in its region, and only that tier goes to the origin. With 200 edges that would each fetch the same new video, a shield tier turns 200 origin fetches into a handful. Concurrent misses for one object at one tier are coalesced into a single fetch. See cache warming and tiered caching.

Origin fetch and freshness#

The CDN fetches with the Cache-Control headers the origin sends. TTLs set how long an object lives, and stale-while-revalidate serves a slightly old copy while one request refreshes it. Large files are fetched in ranges, so a viewer can start before the whole file has been cached.

Purge#

A purge request goes to a control plane that fans it out to every point of presence, usually over a message bus. Purge by URL is simple. Purge by tag, such as everything for one product, needs an index from tags to cached objects. Fast global purge in about a second is a selling point at Fastly and Cloudflare. See CDN push and pull.

Where it breaks#

Cold start after a purge. A global purge empties every edge, and every edge refetches from the origin together. The shield tier and request coalescing absorb most of it. Purge by tag and not everything.

Wrong cache key. Leaving the cookie out of the key can serve one user's account page to another. Never cache responses that set cookies unless the key accounts for them. See the Cached page leak incident.

Long tail objects. Most requests are for popular files, but a vast number of files are asked for once. Those always miss, and they flush useful objects from the cache. Admit an object only on its second request.

Attack traffic. A DDoS aimed at the CDN's own addresses is spread across hundreds of sites by anycast, and a WAF filters what remains. See WAF and bot management.

Build it in Quorum#

The game's CDN is the edge cache. The first board puts a CDN in front of the origin. Look at how the App Server and Database loads fall and how the hit ratio, TTL and purge on deploy trade freshness for hits.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 15.0k Users the Simulation solves this Design to Apdex 0.99, p99 386 ms, 0.0% errors and $7.89/s Burn.
WhereSettingValueWhy
CDNTTL (s)3600Page shells and assets change rarely, so the edge can keep them for the longest time on offer.
CDNPurge on deployOnA release empties the edge, so users do not see the last release's pages until the TTL runs out.
App ServerInstance sizelargeUnchanged. It now sees only what the edge misses.
DatabaseInstance sizelargeUnchanged.

Research this Design needs: Content delivery, Object storage.

The second shows pull, where the edge fills from the origin on a miss, as in the origin fetch step.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 3.00k Users the Simulation solves this Design to Apdex 0.97, p99 491 ms, 0.0% errors and $2.00/s Burn.
WhereSettingValueWhy
CDNTTL (seconds)3600An edge copy is reused for an hour before it asks the origin again, which raises the hit ratio and ages pages longer.
CDNPurge on deployOnA deploy clears the edge, so no visitor gets the old release. The edge then refills from the origin.
CDNCache keyqueryThe key holds the path and query string but not cookies, so more visitors share one copy. It is safe only while no cached page is personal.
App ServerInstance sizesmallSmall gives the app two vCPUs, enough to run code while the data tier does its own work.
DatabaseInstance sizesmallSmall gives the Database two vCPUs, so the figures below reflect the Cache and not a starved Database.

Research this Design needs: Content delivery.

The third is Netflix's Open Connect, a CDN built and placed by the company itself, with a CDN for the bytes and a Cache for personal data.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 4.00k Users the Simulation solves this Design to Apdex 1.00, p99 73 ms, 0.0% errors and $28/s Burn, of which $0.05/s is egress. The board is the request path at game scale. Open Connect’s own appliances and EVCache’s Kafka replication are not on it. The caption is Quorum's result, not the company's numbers.
WhereSettingValueWhy
CDNTTL (seconds)3600An edge copy lives for an hour, as a video object does on an Open Connect appliance.
CDNPurge on deployOnA new release clears the edge, so no viewer gets the old file.
CDNCache keyqueryVideo is not personalized, so the path and query are enough. The personalized pages stay on the app tier.
App ServerInstance sizelargeLarge, because the control plane serves the personalized pages.
App ServerCopies4Four copies stand for the stateless app servers behind the edge.
CacheEnginememcachedEVCache runs on memcached, and memcached is the game engine that scales with cores.
CacheCache strategyasideThe app fills the cache on a miss, the read path EVCache serves.
CacheTTL (seconds)60A minute bounds how stale a personalized answer can be.
CacheInstance sizelargeLarge, because the cache holds each member’s personalized state.
DatabaseInstance sizemediumMedium, because most reads stop at the cache or the edge.
Object StorageStorage classstandardThe origin keeps the video objects that the edge has not cached.
Object StorageLifecycle (days)30Older titles move to a cheaper class after a month.

Research this Design needs: Content delivery, Caching, Object storage, Content delivery, Caching, Object storage, Tiered caching.

The fourth is the routing problem. Look at what far Users pay when they enter one home region, and compare it with Multi-region with GeoDNS.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 3.00k Users the Simulation solves this Design to Apdex 0.91, p99 370 ms, 0.0% errors and $16/s Burn.
WhereSettingValueWhy
App ServerInstance sizelargeLarge, so distance and not capacity is what the far users feel.
DatabaseInstance sizelargeMatch the App Server.

Research Content delivery and Tiered caching, and read Static files on object storage and a CDN. The Premiere spike incident tests an edge under a burst, and the Far Away Sandbox shows distance as the cause of slowness.

The game models one managed CDN and pull only. It has no points of presence you place, no anycast announcements, no purge by tag, and no per-region shield you configure. Tiering is an effect of research and not a topology.

Common questions#

How does a CDN route users to the nearest server?#

Either DNS returns the address of a close point of presence, or every site announces the same anycast address and internet routing delivers each packet to the nearest. Anycast also moves traffic away from a failed site automatically.

What is an origin shield?#

It is an extra cache tier between the edges and your origin. Many edges that miss on the same object ask the shield, and only the shield fetches from the origin.

How does a CDN purge content everywhere?#

A control plane sends the purge to every point of presence, which drops the matching objects. Tagged purges need an index from tags to objects, so that one command can clear a whole group.

What is the difference between push and pull CDNs?#

With pull the edge fetches an object from your origin the first time someone asks. With push you upload files to the CDN yourself, which suits large files that rarely change.