Module 02 · Lesson 05
Single point of failure
One question finds them all: what happens if this dies right now?
What you will be able to do
- Define a single point of failure precisely
- Find SPOFs by walking the request path and asking one question
- Name four SPOFs candidates commonly miss
- Rank SPOFs by blast radius rather than by component size
The test
For each component: if this were gone for ten minutes, could a user still complete the core action? If the answer is no, it is a single point of failure — regardless of how many replicas the diagram shows, because replicas that cannot take over do not count.
Pick the core action
Not every action. For a shortener it is following a redirect. For a store it is completing checkout. SPOF analysis against the wrong action produces a list of everything.
Trace one request, listing every hop
Client, DNS, CDN, load balancer, gateway, auth, app, cache, database, and any synchronous dependency. Write each hop down; the list is usually longer than expected.
Ask the question at each hop
"Gone for ten minutes: can the user still do the core action?" Mark the ones where the answer is no, then check whether recovery is automatic and how long it takes.
Check the control plane too
The deploy pipeline, secret store, feature-flag service and certificate renewal are not on the request path, but they decide whether you can fix anything during an incident.
The usual suspects
| SPOF | Why it is missed | Cheap mitigation |
|---|---|---|
| DNS | Invisible until it breaks; caching hides it in tests | Multiple providers or a secondary zone, low TTL on failover records |
| Load balancer | Drawn as a box everyone assumes is durable | Managed LB, or an active-passive pair with health checks |
| ID / sequence generator | One small service; nobody asks what happens without it | Per-node ID ranges, UUIDs, or a local sequence with a prefix |
| Certificate and secrets | Operational, not architectural | Automated renewal with alerts well before expiry |
| Deploy pipeline | Not on the request path | A documented manual path to roll back without it |
How to say it
"Following the redirect path, the load balancer and the primary database are the two places a ten-minute outage stops redirects entirely. The database has a standby with automated promotion; the load balancer is managed with health checks across two zones. Everything else degrades rather than fails."
Prioritising the list
Every component will feel critical. Rank by two axes instead of by feeling: blast radius — how much of the product stops — and time to recover, which includes noticing. A cache that dies costs latency and is absorbed; an ID generator that dies stops every write until someone intervenes. Same diagram weight, very different priority.
Q. Can a SPOF be acceptable?
Yes, if it is named and priced. A single primary database with a documented 15-minute manual restore may be the right call for an internal tool used by 200 people. The failure mode is pretending it is not there; the fix is saying "this is our one SPOF, here is the recovery time, and here is the cost of removing it."
Q. Does adding a replica always remove a SPOF?
Only when three things hold: independent failure domains, automatic detection, and automatic traffic shifting — plus verified behaviour under real failure. Replicas in one zone with manual promotion turn a fast failure into a slow one, which is not the same as removing the SPOF.
Worth remembering
- A SPOF is any component whose failure stops the user from completing the core action — including components with no replica.
- Walking one request path end to end finds more SPOFs than staring at the architecture diagram.
- DNS, the load balancer, the ID generator, the certificate and the deploy pipeline are the usual missed ones.
- Rank by blast radius and recovery time, not by how important the component sounds.
Checkpoint
Answer all 2 questions to finish the lesson.
1. Which is a component whose ten-minute failure stops the core user action?
A SPOF is defined by whether the core action survives. Reads can fall back to the primary and analytics can lag; writes stop entirely without IDs.
2. How should SPOFs be prioritised?
What matters is how much of the product stops and how long it stays stopped, including detection time. Component size says nothing about either.