Module 01 · Lesson 04
Numbers to know cold
The dozen figures that let you size a system without hesitating.
What you will be able to do
- Recite the latency of memory, SSD, disk, and network round trips by order of magnitude
- Convert availability targets into allowed downtime per month and per year
- State the throughput of a single typical database, cache, and message broker
- Use a latency budget to decide where a request may call out
- Correct a wrong number in the room without losing the thread
Why memorise anything
Hesitating on "is 5,000 writes a second a lot?" costs you the authority of every decision that follows. A dozen figures, correct to the nearest power of ten, are enough to size almost any design and to reject a bad one on the spot.
Latency: the ladder
| Operation | Latency | What it means for a design |
|---|---|---|
| L1 cache reference | ~1 ns | Free. Ignore it. |
| Main memory read | ~100 ns | Free. In-process caching costs nothing. |
| Compress 1 KB | ~10 µs | Cheap enough to do per request. |
| Read 1 MB sequentially from memory | ~250 µs | Cheap. In-memory joins are fine. |
| Round trip within one datacenter | ~0.5 ms | Service-to-service calls are cheap but not free. |
| Read 1 MB from SSD | ~1 ms | A database page read is on this order. |
| Read 1 MB from disk | ~20 ms | Disk is for cold data only. |
| Disk seek | ~10 ms | Random I/O is the enemy; this is why LSM trees exist. |
| Round trip cross-continent | ~100-150 ms | One user-visible delay. Never in a request loop. |
The ladder is the point: each rung is roughly 1,000x the one below. Memory to SSD is 1,000x, SSD to a cross-region round trip is 100x. That is why caching in memory works at all, and why a synchronous call to another region inside a request is a design smell.
The 100 ms rule
Above roughly 100 ms a person notices a delay; above a second they lose their train of thought. So a page render has a budget of a few hundred milliseconds, and any dependency that costs 100 ms can appear in it at most once.
Availability: what each nine buys
| Target | Per year | Per month | What it takes |
|---|---|---|---|
| 99% | 3.65 days | ~7 hours | One server and hope |
| 99.9% | 8.8 hours | ~43 minutes | Redundancy, health checks, a real deploy process |
| 99.95% | 4.4 hours | ~22 minutes | Multi-AZ, automated failover, alerting |
| 99.99% | 53 minutes | ~4.3 minutes | Multi-region or very mature single-region operations |
| 99.999% | 5 minutes | ~26 seconds | Active-active regions and a team that does little else |
Note the shape: each additional nine cuts allowed downtime by 10x and typically multiplies cost by a similar factor. In an interview, question the target before designing for it — "99.99% across regions is expensive; is 99.9% with a stated recovery time acceptable?" is a senior question.
Throughput: one node of each thing
Postgres / MySQL node
1k-5k writes/s
Reads with replicas: 10k-50k/s. Numbers drop fast with joins and large transactions.
Redis node
~100k ops/s
Single-threaded, sub-millisecond. The gap to a database is why caching wins.
Kafka broker
~100MB/s-1GB/s
Sequential writes; scale by partitions and brokers, not by faster disks.
Elasticsearch node
~1k-5k queries/s
Depends heavily on query shape and index size.
One app server
~1k-10k req/s
For simple, mostly-I/O-bound handlers. Less as soon as you do real work.
A single TCP connection
~1 Gbps
So 100 MB/s is a hard ceiling per connection — relevant for uploads.
The safe escape
If you cannot recall a figure, give a range and the shape: "somewhere in the low thousands of writes a second per node, so we need roughly ten for 20k/s." A stated range with the arithmetic attached is more credible than a confident wrong number.
A latency budget, worked
Target: a 300 ms page load with 20 backend calls' worth of data to gather. Spend the budget explicitly:
Budget: 300 ms end to end network to the edge 50 ms gateway + auth 10 ms cache lookups (x3) 3 ms database (on miss) 20 ms fan-out to 5 services 50 ms (in parallel: 10 ms each) serialise 200 KB 10 ms ------------------------------ used 143 ms -> 157 ms of headroom Twelve services called in sequence at 10 ms each would have been 120 ms -- the budget only works because they run in parallel.
That table is a stronger answer than any diagram detail, because it forces two conclusions out loud: parallelise independent reads, and cache to avoid the database path entirely on the common case.
Using the numbers without showing off
Instead
Reciting numbers nobody asked for: "L1 cache is 0.5 ns, main memory 100 ns, SSD..."
Do this
Deploy a number when it decides something: "a 100 ms cross-region call cannot be in the critical path, so replicas are region-local."
Instead
Quoting a number with false precision: "Redis does 109,000 operations per second"
Do this
"roughly 100k ops per second on one node" — the order of magnitude is the claim you can defend.
Instead
Designing for five nines because it sounds impressive
Do this
Ask what availability is worth paying for, then design to that. Overpromising availability is a red flag, not a strength.
Instead
Silently disagreeing with the interviewer's figure
Do this
"I had it at about 1,000 writes per second per node — happy to use your number; either way we need about ten nodes." Disagree on the input, keep the conclusion.
Q. Do I need to memorise the exact latency table?
No. Memorise the ladder: nanosecond, microsecond, millisecond, hundred-milliseconds. If you know memory is nanoseconds, SSD is microseconds, disk is milliseconds and inter-region is a hundred milliseconds, every design argument you need can be made.
Q. What if the interviewer says my throughput assumption is wrong?
Thank them and rerun the conclusion in one sentence: "If it is 10k writes a second instead of 2k, one node becomes five and we need the partitioning discussion now." The value you demonstrated was the ability to re-derive, not the memorised figure.
Worth remembering
- Memory is 100 ns, SSD read is 100 µs, disk seek is 10 ms, a cross-region round trip is 100 ms. Each step is roughly 1000x the one before.
- Every extra nine of availability costs about 10x the engineering. 99.9% is 43 minutes of downtime a month; 99.99% is 4 minutes.
- One Postgres node handles low thousands of writes per second; one Redis node handles ~100k operations per second. That gap decides where you cache.
- Latency is a budget: a 200 ms page request cannot afford ten sequential 50 ms calls.
Checkpoint
Answer all 3 questions to finish the lesson.
1. How much downtime does 99.9% availability allow per month?
99.9% leaves about 0.1% of a month, which is roughly 43 minutes. 99.99% is about 4 minutes, and 99% is about 7 hours.
2. A request must render in 200 ms and needs data from six independent services, each taking 15 ms. What makes it fit?
Parallel fan-out makes the cost the slowest call rather than the sum. Sequentially it is 90 ms of the 200 ms budget before you have serialised anything.
3. Why does a cache in front of a database change the design so much?
The ladder is the reason: a memory read is ~100 ns and a disk seek ~10 ms, three orders of magnitude apart. A cache hit removes almost the whole database cost, which is why the read path is usually redesigned around it.