Module 02 · Lesson 01
Scalability
Growth is the default state. Scalability is whether your design survives it.
What you will be able to do
- Distinguish vertical from horizontal scaling and name the ceiling of each
- Explain why stateless services scale and stateful ones do not
- Identify the three bottlenecks that survive adding more app servers
- Choose a scaling axis for a stated workload
Two directions, one ceiling each
| Vertical (bigger box) | Horizontal (more boxes) | |
|---|---|---|
| What you do | More CPU, RAM, faster disks on one machine | Add more machines behind a load balancer |
| Ceiling | Largest instance money can buy | Near-linear, until a shared dependency appears |
| Cost curve | Super-linear — 2x the machine is often 4x the price | Linear, in both money and operational complexity |
| Failure mode | One machine, one blast radius | Partial failure: some nodes healthy, some not |
| Code changes | None | Requires statelessness and shared-state design |
Vertical scaling is the honest first move: a single well-tuned Postgres on a large machine carries a surprising amount of traffic, and there is no distributed-systems tax. Interviewers respect "I would buy a bigger box first, and here is the number where that stops working" more than premature sharding.
Say the number where it stops
"One primary node handles low thousands of writes a second; we need about 1,200 at peak, so a single node holds. I would plan the shard key now and partition when we pass ~3,000." A scalability answer without a breakpoint is just optimism.
Statelessness is the enabler
A stateless service keeps nothing between requests that the next request needs. Any node can serve any request, so the load balancer is free to send traffic anywhere, and a dead node costs in-flight requests only. That single property is what makes horizontal scaling boring.
- Sessions move out of process memory into a shared store (Redis) or a signed cookie.
- Uploaded files move to object storage, not the local filesystem.
- Caches become a shared tier, so a cache miss on one node does not mean a cold node.
- Background jobs move to a queue with workers that have no affinity to a specific user.
Where scaling actually stops
The shared database
Ten app servers all hitting one primary is still one primary. Fixes, in order of cost: indexes and query shape, caching, read replicas for read-heavy loads, then partitioning for write-heavy ones.
The hot key
Uniform hashing breaks when one key is far more popular than the rest — a celebrity's profile, a flash-sale item. Fix by splitting the hot key into sub-keys with an aggregator, or by caching that key aggressively.
Synchronous fan-out
A request that calls eight services sequentially multiplies its latency by eight and its failure probability by more. Fix: parallel calls with timeouts, or move the work to a queue when the user does not need the result immediately.
Coordination
Global counters, distributed locks and unique-ID generation all serialise somewhere. Fix: batch allocation (hand out ID ranges per server) or accept approximate counts.
Q. Is a single database ever the right answer at scale?
Yes, far more often than candidates assume. A tuned Postgres with a couple of replicas and a cache in front serves most products. The right answer is usually "one primary until write volume or dataset size forces partitioning, and here is the threshold" rather than reaching for a distributed store on day one.
Q. What is the first thing you would check when a system slows at 3x traffic?
Whether it slowed linearly or collapsed. Linear degradation points at a shared resource sustaining its ceiling — connection pool, CPU, lock contention. A cliff points at something with a fixed limit: a queue filling, a cache eviction storm, a connection limit, or a hot key.
Worth remembering
- Vertical scaling buys time with money; horizontal scaling buys capacity with complexity. Both end somewhere.
- App servers scale almost linearly once they hold no state. The database is what stops you.
- Adding servers to a system with shared counters or in-memory sessions creates a correctness problem, not capacity.
- Name the axis you are scaling — reads, writes, storage, connections — because each has a different fix.
Checkpoint
Answer all 2 questions to finish the lesson.
1. What property must a service have before you can add more instances to handle load?
Without state kept between requests, any instance can serve any request. Sessions, local files and per-node caches all break that assumption and force sticky routing.
2. You add five more app servers and throughput barely moves. Most likely cause?
App servers stop helping once the next shared tier saturates. The ceiling moves to the database, a lock, or a hot key — that is where the next fix belongs.