Module 03 · Lesson 02
TCP, UDP, and the need for push
Most designs need a request and a reply. Some need the server to speak first.
What you will be able to do
- Explain what TCP guarantees that UDP does not
- Pick a push mechanism for chat, live scores, and notifications
- Estimate the connection cost of keeping a million sockets open
- Describe how presence and reconnect logic change a design
What each transport promises
| TCP | UDP | |
|---|---|---|
| Delivery | Retries until acknowledged | Best effort, may vanish |
| Ordering | Guaranteed in order | None |
| Head-of-line blocking | Yes — one lost packet stalls the stream | No — later packets keep flowing |
| Connection state | Handshake, tracked per connection | Connectionless |
| Good for | APIs, databases, file transfer, chat messages | Voice, video, games, telemetry |
The interesting column is head-of-line blocking. TCP will not hand you packet 12 until packet 11 arrives, because your application asked for a stream. For a chat message that is exactly right. For a video call, waiting for a retransmitted frame produces a freeze, while skipping it produces a small glitch — so real-time media runs on UDP and tolerates loss.
QUIC, briefly
QUIC runs over UDP but rebuilds reliability, ordering, and encryption in user space, per stream. That gives it two practical wins: one lost packet blocks only its own stream, and a new connection can resume without a full handshake. HTTP/3 is HTTP over QUIC. In a design round, mentioning it as an option for mobile or lossy networks is enough.
Push: four ways to get data to a client
- Polling — the client asks every N seconds. Trivial to build, wasteful, and adds latency up to N seconds. Fine for a dashboard refreshing every minute.
- Long polling — the server holds the request open until there is something to say. Works through most proxies, but each message costs a new request.
- Server-sent events — one HTTP connection, server writes text events forever, client reads. Perfect for feeds, scores, and notifications where the client only listens.
- WebSocket — a single connection upgraded from HTTP where both sides send frames at any time. The default for chat, collaboration, and multiplayer.
Choose by direction and volume. If the client mostly listens, SSE is simpler than a socket and reconnects by itself. If both sides talk constantly, use a socket. If updates are rare, polling is honest and cheap.
What a million connections costs
Memory per idle socket
~10-50 KB
Kernel buffers plus your per-connection state
1M sockets
10-50 GB
Before any application data
File descriptors
1M+
Default limits are far lower
Heartbeat interval
30-60 s
Detects dead clients and keeps NAT mappings alive
The design consequence: you do not run one server for a million sockets. You run many WebSocket servers, each holding a bounded number, and you make routing explicit — a registry maps user to server, usually in Redis or a coordination service, so any node can deliver a message to the right connection. That routing table is the part interviewers want to hear about.
Connect once, authenticate at upgrade
Validate the token during the WebSocket handshake; do not accept an unauthenticated socket and ask later.
Register the connection
Write user id to node id into a shared registry with a short TTL, refreshed by heartbeat. The TTL is what cleans up after a crashed server.
Route through a message bus
Producers publish to a per-user topic; the node holding that user's socket consumes and writes the frame. This decouples your API from your connection fleet.
Handle reconnect explicitly
Store a cursor per user so a reconnecting client can fetch what it missed instead of losing messages silently.
The thing candidates forget
A WebSocket is state on the server. Load balancers must not shuffle a connection between nodes, deploys must drain sockets gracefully, and a node failure disconnects everyone on it. Say how clients reconnect and how you avoid a thundering herd when a node restarts.
Worth remembering
- TCP gives ordered, reliable, congestion-controlled byte streams; UDP gives small messages with no promises.
- Use UDP when a late packet is worse than a lost one — voice, video, real-time games.
- Push usually means a long-lived connection: WebSocket for two-way, SSE for server-to-client only, polling as the fallback.
- A million idle connections is a file-descriptor and memory problem, not a bandwidth problem.
Checkpoint
Answer all 3 questions to finish the lesson.
1. A live video call drops a packet. Why is UDP usually the better transport?
TCP's head-of-line blocking makes the receiver wait for the lost packet. In real-time media, a slightly glitchy stream beats a frozen one.
2. A notifications panel needs server-to-client updates and nothing else. Simplest suitable mechanism?
SSE is one long-lived HTTP response written by the server, with automatic reconnect. A bidirectional socket is more machinery than a one-way feed needs.
3. Supporting 1M concurrent idle WebSocket connections is primarily limited by:
Idle sockets consume kernel and application memory plus a descriptor each. That is why connection fleets are sharded across many nodes with an explicit routing registry.