Module 12 · Lesson 06
Design a chat application
Durable connections, ordered messages per conversation, and delivery that survives a phone going offline.
What you will be able to do
- Design a connection layer that scales beyond one gateway
- Guarantee message order within a conversation
- Deliver messages to offline and multi-device users
- Avoid a write storm from presence and receipts
Requirements and estimates
- Functional: one-to-one and group messaging, delivery and read receipts, presence, offline sync, media attachments.
- Non-functional: p99 delivery under 200 ms for online users, no duplicate or out-of-order messages in a conversation, history retained indefinitely.
- Scale: 50M concurrent connections, 10B messages/day (~115k/s average, 500k/s peak).
50M connections / 50k per gateway = 1,000 gateway nodes
10B msgs/day x 200 B = 2 TB/day of message metadata
(media in object storage, ~20x more bytes)
fan-out: group of 500 -> one message becomes 500 deliveriesConnection layer
Clients hold a WebSocket to a gateway. The gateway registers user_id -> gateway_id in a shared registry with a short TTL and heartbeat. When a message arrives for that user, the sender's gateway looks up the recipient's gateway and forwards it over an internal channel. If the lookup misses, the user is offline and the message stays in the durable store for later sync.
Reconnection storms are the real outage
A gateway restart drops 50k connections that all retry at once. Clients need exponential backoff with jitter, and the gateway should shed and rebalance connections gradually.
Ordering and storage
Assign a sequence
Partition by conversation id; one owner assigns monotonically increasing seq numbers.
Persist before ack
Write the message, then acknowledge the sender. An ack for unpersisted data loses messages on crash.
Fan out deliveries
Push to online recipients; for groups larger than a threshold, fan out through a log rather than inline.
Advance cursors
Each device stores the highest seq it has received and requests everything above it on reconnect.
Per-conversation sequence solves ordering and gap detection at once: if a device receives seq 12 after 15, it knows it missed 13 and 14 and can request them. That is far simpler than relying on timestamps across devices.
Receipts, presence and media
- Read receipts: coalesce per conversation. Update the highest read seq, not one event per message.
- Typing indicators: ephemeral pub/sub only, never persisted, dropped under load.
- Presence: heartbeat with a TTL in Redis; derive online state from TTL rather than writing every transition.
- Media: upload directly to object storage with a signed URL, then send the reference in the message.
- Multi-device: each device is an independent cursor; the server never assumes one active client.
One paragraph that lands
"Gateways hold WebSockets and register user-to-gateway in a TTL registry. Messages are persisted with a per-conversation seq before ack, delivered to online devices, and synced from each device's cursor on reconnect. Receipts are coalesced and presence is TTL-derived."
Worth remembering
- Connections are stateful, so routing a message to a user requires a registry of which gateway holds them.
- Order is per conversation, assigned by a single owner of that conversation's sequence.
- Offline delivery is a per-device cursor over a durable message log, not a queue that forgets.
- Typing indicators and presence are ephemeral: drop them under load rather than persisting them.
Checkpoint
Answer all 3 questions to finish the lesson.
1. Why acknowledge a message only after persisting it?
The sender stops retrying after an ack. If the write had not committed, the message is gone with no retry.
2. How does a device detect it missed messages while offline?
A monotonic seq per conversation makes gaps explicit; the device requests everything above its last cursor.
3. What is the correct treatment of typing indicators?
They are transient signals with no history value. Persisting them adds write load for data nobody reads later.