QuorumWiki
Jugar
Guías prácticas

Diseñar un servicio de streaming de vídeo

Diseña un servicio de streaming de vídeo para una entrevista de diseño de sistemas: subida, cola de transcodificación, almacenamiento de objetos, CDN y bitrate adaptativo, y lo que enseña Quorum.

A video service has two paths. Uploads go to object storage, then a queue feeds a fleet of transcoding workers that cut each video into short segments at several quality levels. Viewers never touch your servers for video: a CDN serves the segments, and the player switches quality as bandwidth changes.

Requirements#

Functional:

  • Creators upload videos of up to several gigabytes.

  • Viewers play a video within a couple of seconds on any device and connection.

  • Search and metadata, such as title and view count, work alongside playback.

Non-functional:

  • Playback starts in under 2 seconds with no rebuffering on a normal connection.

  • Uploads survive a dropped connection.

  • Storage is durable. A lost original cannot be recreated.

  • Egress is the largest cost, so the design must keep traffic off the origin.

Estimate the load#

Assume 100 million daily viewers, each watching 5 videos of 5 minutes, and 500,000 uploads a day at 5 minutes each.

  • Egress: 100M x 5 x 5 min = 2.5 billion minutes a day. At an average of 3 Mbps, that is 3 Mbps x 60 s / 8 = 22.5 MB per minute, so 2.5B x 22.5 MB = 56 PB a day. Peak concurrent viewers are about 2.5B / 1,440 x 3, roughly 5 million streams, or 15 Tbps.

  • Upload storage: a 5-minute original at 1080p is about 300 MB, so 500,000 x 300 MB = 150 TB a day. Transcoded renditions add about 2x.

  • Transcoding: 500,000 videos a day is about 6 a second, and each takes several CPU minutes per rendition.

No origin can serve 15 Tbps, so the CDN is the design. The write side is small by comparison.

The design#

Upload#

The client splits the file into chunks and uploads each directly to Object Storage with a pre-signed URL. The App Server only issues the URL and records metadata. Direct upload keeps a multi-gigabyte file from holding an App Server slot, and chunks let a failed upload resume. See claim check.

Transcoding queue#

When the upload completes, a message goes to a Queue. Workers pull a job, split the video into segments of a few seconds, and encode each segment at several resolutions and bitrates, for example 240p to 4K. Segments encode in parallel, so a two-hour film finishes in minutes. This is slow work to a queue and workers. Jobs can run twice, so write outputs under deterministic keys.

Storage layout#

Originals stay in Object Storage in a cold class. Rendition segments and a manifest file per video sit beside them. A metadata database holds title, owner, status, and the manifest location. Lifecycle rules move old, unpopular videos to cheaper classes.

CDN#

The CDN sits first in the viewing path. It caches segments at edge locations near the viewer and falls back to a regional cache, then to Object Storage. Video is highly cacheable and popularity is skewed, so the hit ratio is very high. Read static files on object storage and a CDN and CDN push and pull.

Adaptive bitrate#

The manifest, in HLS or DASH format, lists every rendition and its segments. The player downloads a segment, measures how long it took, and picks the next segment's quality accordingly. A throttled phone drops to 360p instead of stalling.

Where it breaks#

Premiere spike. A hit release sends millions of viewers to the same segments in the same minute. A cold edge misses on all of them, and the misses stack up on the origin. Pre-warm the CDN, add a mid-tier cache that collapses duplicate misses, and shed load before the origin falls over. The game incident is Premiere spike, and cache warming and tiered caching is the fix.

Transcoding backlog. A burst of uploads leaves the queue growing for hours, and creators wait to publish. Scale workers on queue depth, and put long jobs on cheap interruptible capacity such as spot instances. See queue load leveling and autoscaling.

Poison jobs. A corrupt file crashes the encoder every time, and an unlimited retry blocks the queue. Cap attempts and park the job in a dead letter queue. The game version is Poison message.

Build it in Quorum#

Three boards cover the viewing path and the storage under it. The first takes bytes off the Database. An App Server sends files to Object Storage, and the Database keeps rows only. Look at the Database's load bar and p99, which fall once files leave its rows. Research Object storage to place it. This is the upload and originals half of the design, and the board is stateless app servers at work.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 15.0k Users the Simulation solves this Design to Apdex 0.80, p99 1.2 s, 4.8% errors and $7.44/s Burn.
WhereSettingValueWhy
App ServerInstance sizelargeUnchanged from the Design before, so the only difference is where the files go.
DatabaseInstance sizelargeUnchanged, and it now stores rows only.
Object StorageProducts3S3 needs no extra research. Uploads and downloads go here instead of into Database rows.

Research this Design needs: Object storage.

The second board puts a CDN in front of the App Server, pulling from the origin on a miss. It has a long TTL and purge on deploy, so the edge refills after a release. Look at how much traffic the App Server and Database see compared with a run that has no CDN. Research Content delivery first. See CDN push and pull, and note that the game models pull only.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 3.00k Users the Simulation solves this Design to Apdex 0.97, p99 491 ms, 0.0% errors and $2.00/s Burn.
WhereSettingValueWhy
CDNTTL (seconds)3600An edge copy is reused for an hour before it asks the origin again, which raises the hit ratio and ages pages longer.
CDNPurge on deployOnA deploy clears the edge, so no visitor gets the old release. The edge then refills from the origin.
CDNCache keyqueryThe key holds the path and query string but not cookies, so more visitors share one copy. It is safe only while no cached page is personal.
App ServerInstance sizesmallSmall gives the app two vCPUs, enough to run code while the data tier does its own work.
DatabaseInstance sizesmallSmall gives the Database two vCPUs, so the figures below reflect the Cache and not a starved Database.

Research this Design needs: Content delivery.

The third board is how Netflix ran it. The CDN stands for Open Connect, a cache close to the viewer that answers media without the origin. The App Servers are the control plane, a Cache holds personal data, the Database is the source of truth, and Object Storage holds the video the edge has not cached. Look at which Component answers the bulk of requests. Own the edge tells the story.

Drag to pan. Hold Ctrl or Cmd and scroll to zoom.

At 4.00k Users the Simulation solves this Design to Apdex 1.00, p99 73 ms, 0.0% errors and $28/s Burn, of which $0.05/s is egress. The board is the request path at game scale. Open Connect’s own appliances and EVCache’s Kafka replication are not on it. The caption is Quorum's result, not the company's numbers.
WhereSettingValueWhy
CDNTTL (seconds)3600An edge copy lives for an hour, as a video object does on an Open Connect appliance.
CDNPurge on deployOnA new release clears the edge, so no viewer gets the old file.
CDNCache keyqueryVideo is not personalized, so the path and query are enough. The personalized pages stay on the app tier.
App ServerInstance sizelargeLarge, because the control plane serves the personalized pages.
App ServerCopies4Four copies stand for the stateless app servers behind the edge.
CacheEnginememcachedEVCache runs on memcached, and memcached is the game engine that scales with cores.
CacheCache strategyasideThe app fills the cache on a miss, the read path EVCache serves.
CacheTTL (seconds)60A minute bounds how stale a personalized answer can be.
CacheInstance sizelargeLarge, because the cache holds each member’s personalized state.
DatabaseInstance sizemediumMedium, because most reads stop at the cache or the edge.
Object StorageStorage classstandardThe origin keeps the video objects that the edge has not cached.
Object StorageLifecycle (days)30Older titles move to a cheaper class after a month.

Research this Design needs: Content delivery, Caching, Object storage, Content delivery, Caching, Object storage, Tiered caching.

Research Tiered caching to raise how much video the edge answers, and run the Premiere spike incident against it. For uploads, link the App Server to a Queue and a Worker, and link the Worker to Object Storage. The Slow Uploads Sandbox teaches that path.

The game has no transcoding step, renditions, or manifests, and a Worker stands for any background job and not for an encoder. Adaptive bitrate is not modeled. The upload path, the storage class, and the CDN hit ratio are the parts you can build and measure.

Common questions#

How does YouTube store and serve videos?#

Originals and transcoded renditions sit in distributed object storage, with metadata in a separate database. Playback comes from CDN edge caches, and only misses reach storage.

What is adaptive bitrate streaming?#

The video is encoded at several qualities and cut into segments of a few seconds. A manifest lists them, and the player picks each next segment's quality from the bandwidth it just measured. Playback keeps going at lower quality instead of freezing.

Why use a queue for video transcoding?#

Transcoding takes minutes of CPU per video and uploads arrive in bursts, so the web tier cannot do it inline. A queue lets uploads return at once while workers drain jobs at their own pace, and retries are safe.

How do you stop a viral video from overloading the origin?#

Cache segments at the edge, put a shared mid-tier cache between edges and storage so duplicate misses collapse into one request, and warm the cache for known releases.