Module 12 · Lesson 09
Design a video platform
Upload is a pipeline; playback is a caching problem. Treat them as two systems.
What you will be able to do
- Separate the upload, transcode and playback paths
- Choose a transcoding fan-out and storage layout
- Design adaptive bitrate delivery through a CDN
- Keep view counting off the critical path
Requirements and estimates
- Functional: upload, transcode to multiple resolutions, stream with adaptive bitrate, view counts, searchable metadata.
- Non-functional: start playback within 2 s of a manifest request, survive a lost CDN region, uploads resumable.
- Scale: 500 hours uploaded per minute, 100M daily viewers.
upload: 500 h/min = 30,000 h/day source ~5 GB/h -> 150 TB/day raw renditions: 5 profiles x ~0.6 of source on average = ~450 TB/day playback: 100M viewers x 1 h at 3 Mbps = ~37 PB/day egress => CDN offload above 95% is the entire business case
Upload and transcode
Request an upload URL
The backend issues a signed, resumable upload URL scoped to a video id and a size limit.
Upload in chunks
The client uploads directly to object storage; retries resume from the last acknowledged chunk.
Publish a job
On completion, an event triggers the transcode workflow with the source location.
Fan out renditions
Split the source into segments, encode each segment per profile in parallel, then package manifests.
Publish and index
Mark the video ready, write metadata, and invalidate the metadata cache.
Segment-level parallelism is what makes transcoding fast: a two-hour video becomes hundreds of short jobs instead of one long one, and a failed segment is retried without redoing the entire file. Store segments by (video id, rendition, segment number) so the manifest is a list of URLs rather than a database join.
Transcoding is a cost lever
Pre-encoding every profile for every video wastes money on long-tail content. Popular videos get all renditions eagerly; the rest are encoded on first play or at lower priority.
Playback
The player requests a manifest, gets a list of segment URLs per bitrate, and adapts to measured bandwidth. Segments are small (2-10 s) so switching is quick. Everything after the manifest is CDN traffic; the origin sees only cache misses. Signed URLs with a short TTL prevent hotlinking without breaking a CDN's ability to cache.
Segment length
2-10 s
Shorter adapts faster, more requests
Renditions
4-6 profiles
240p to 4K, ladder by source quality
CDN hit ratio target
> 95%
Below this, egress cost dominates
View count flush
every 30-60 s
Aggregated in memory, then persisted
Metadata and counts
- Metadata: video id, title, owner, duration, renditions, status. Cached aggressively, invalidated on publish.
- Search: a separate index over titles, tags and transcripts, updated asynchronously.
- View counts: buffer increments in memory or Redis and flush in batches; the UI shows an approximate number.
- Exact analytics: a stream of play events to a warehouse, where counting is someone else's query.
Frame the split
"Upload is a resumable signed URL into object storage, then a segment-parallel transcode workflow. Playback is manifests plus short segments served by the CDN, with signed URLs. Views are aggregated and flushed, never a per-play database write."
Worth remembering
- Uploads go straight to object storage with a signed URL; the API server should not proxy bytes.
- A single uploaded video becomes many renditions, each segmented into small chunks.
- Playback is CDN-served static content; the origin only signs URLs and serves manifests.
- View counts are approximate first, exact later — never a synchronous database update per play.
Checkpoint
Answer all 3 questions to finish the lesson.
1. Why should uploads go directly to object storage?
Signed direct uploads keep multi-gigabyte transfers off the API tier, which would otherwise be bandwidth- and memory-bound.
2. What is the benefit of segment-level transcoding?
Small independent jobs encode in parallel and fail independently, instead of one long job that restarts from zero.
3. How should view counts be maintained?
A write per play makes the hot path expensive for a number that only needs to be approximate.