Design doc · 09 of 12

YouTube end to end

A full video platform: path routing, a real-world metadata cache, async uploads and transcoding, view counting, and capacity limits.

Template
YouTube end to end
Level
Advanced
Reading time
25 min
Topics
Path routingCDNSharded SQLCache sizingAsync pipelinesKafkaCapacity planning

The challenge in the lab 20,000 requests/s routed by path: watch, search, comments and uploads. Metadata cache in front of Vitess MySQL, recommendations, view events into Kafka, uploads to storage and a transcoding queue. Keep success ≥ 99%, p99 ≤ 500 ms and Apdex ≥ 0.90. Then push it: 3x traffic, a DDoS, a cache flush. Find what breaks first and fix it.

1Context and scope

People watch, search, comment and upload. Video bytes come from a CDN; this design handles the API traffic around them. Watches dominate, uploads are rare but heavy, and every path has different needs, so the gateway routes each request type to its own service.

2Goals and non-goals

Goals

  • 20,000 API requests a second: 80% watch, 12% search, 6% comment, 2% upload.
  • Success at least 99%, p99 at most 500 ms, Apdex at least 0.90.
  • Uploads processed asynchronously with nothing lost.

Non-goals

  • Video bytes themselves: the CDN serves segments; this design is the API traffic around playback.
  • Live streaming, ads and copyright matching.
  • Multiple regions and data residency.

3Capacity estimate

Back-of-the-envelope numbers from the template. The simulation uses the same ones, so you can check them against a run.

QuantityValueHow
Served at the CDN≈ 6,000/s30% of reads are cacheable pages
Watch API≈ 11,200/seach reads metadata, ranks recommendations, emits a view event
Metadata cache64 GB, 20M videos, ~91% hitspopularity skew keeps hot videos in memory
Metadata reads on MySQL≈ 1,000/smisses only
Upload chunks≈ 400/s2 s to transcode each: about 800 transcoders busy
Capacity≈ 40,000 req/s2× headroom; the Watch API gives way first

4The design

Each component in the template, in the order a request meets it, and why it is there.

  1. DDoS protection

    Absorbs floods before the CDN and gateway.

  2. Media CDN

    Serves cacheable reads (and, in reality, the video bytes) at the edge.

  3. API gateway with routes

    Path routing: /watch, /search, /comment and /upload each go to their own service, so they scale and fail independently.

  4. Watch API

    The hot path: metadata, recommendations, a view event. Sized for about 2× headroom.

  5. Metadata cache (Redis)

    Real-world sizing: 64 GB for 20 million videos; Zipf popularity gives a high hit rate.

  6. Video metadata (Vitess MySQL)

    Sharded MySQL with read replicas; sees only cache misses and writes.

  7. Recommendations (gRPC)

    A ranking microservice called on every watch: it scores candidate videos for the viewer. gRPC keeps the internal call small and fast.

  8. View events (Kafka) and view counter

    Counting views asynchronously keeps the watch path fast; the counter writes to a Bigtable-style store.

  9. Search API and index

    Inverted index for queries; separate from the watch path.

  10. Comments API and store

    A document store for comment threads.

  11. Upload API, storage, queue, transcode workers and encoder farm

    Chunks are stored and a transcode job is queued. Transcode workers take each job, have the encoder farm cut it into renditions (1080p, 720p, ...), and write each one back to storage.

5Key concepts

Separate the paths

Watch, search, comment and upload have different traffic, latency and cost. Routing them to separate services lets each scale on its own and stops a slow path from hurting the others.

Asynchronous where you can

Uploads and view counts do not need to finish before the user gets a response. Put them on queues and streams; the user-facing path stays fast and bursts are absorbed.

Size caches from the real numbers

Memory, item size, key count, popularity and TTL decide the hit rate. Popular videos are a small share of all videos, so a cache that holds a fraction of them still answers most reads.

Headroom and the first thing to break

Capacity planning keeps about 2× headroom. A stress test shows which component gives way first (here the Watch API) so you know what to scale before a launch.

6Alternatives considered

  • More cache memory and longer TTLs raise hit rates but cost money and serve older data.
  • Async pipelines are resilient but add delay before uploads are watchable and counts are current.
  • Path routing adds services to run, in return for independent scaling.

7Failure modes and fixes

What goes wrong in the simulation, why, and the change that fixes it. Each one shows up as a finding in the lab; ask “what broke?” to see it, and for the fix when you want it.

At 3× traffic the Watch API saturates and the transcode queue backs up.

WhyThe Watch API was sized for 2× and the transcoder fleet for normal uploads.

FixScale the Watch API (12 → 18 replicas) and double the transcoders; the simulation shows 99.9% again.

The stress test finds no defence against bots or credential stuffing.

WhyOnly DDoS protection sits in front.

FixAdd a WAF and bot management between the CDN and the gateway.

8Try it in the lab

Open the template, then type these into the copilot one at a time. The first few are also offered as buttons when the template opens.

  1. 3x traffic

    The Watch API saturates; the live suggestions offer more replicas.

  2. launch a ddos attack

    The shield blocks about 98%; users are unaffected.

  3. flush the cache

    Metadata reads move to MySQL; watch how the replicas cope.

  4. stress test

    Capacity about 40,000 req/s, the Watch API first to give way, and the missing bot defence.

  5. add a waf

    Closes the credential-stuffing gap the stress test found.

Open “YouTube end to end” in the lab

9Check yourself

  1. Why route /upload to its own service instead of the Watch API?
  2. Why count views through Kafka rather than writing them on the watch request?
  3. Which component would you scale first before a 2× launch, and how would you know?

10Further reading