System design · scaling data
When one machine stops being enough, you have three moves: shard the data, replicate it, or cache it. This sheet shows, for every real database, how you scale it for reads vs writes, what you reach for, and the trade-off you take on. Companion to S4 · Data at Scale: Sharding, Replication, Caching.
Every scaling move removes a limit and adds a bill. Climb the cheap rungs first (bigger box, replica, cache) and shard only when forced. The senior skill is not the trick, it is naming what you bought and what you now owe: a new failure mode and a “this might be slightly stale.”
Before any tool, diagnose. The fix depends entirely on whether too many reads or too many writes are the pain. Get this wrong and you open a one-way door you did not need.
Pages are slow, the DB is busy serving the same queries. Fix with read replicas and a cache. Cheap, reversible, huge wins.
One machine can’t absorb the write rate or hold the data. Bigger box first, then shard. Only sharding truly scales writes.
Sharding a read problem. Most teams that “need to shard” had a reads problem a replica or a cache would have fixed in an afternoon, without the permanent cost of a shard key.
Climb in order. Each rung is cheaper and less risky than the next. Do not jump to the top.
| Rung | Move | Helps | New cost |
|---|---|---|---|
| 1 | Bigger box (vertical) | reads & writes | none, until the ceiling; then it stops |
| 2 | Read replicas | reads | replica lag, stale reads |
| 3 | Cache | reads | invalidation, staleness, stampedes |
| 4 | Shard (horizontal) | writes | one-way door, cross-shard queries hard |
Vertical is boring and frequently correct. A read replica or cache buys years. Sharding is the last resort, not the first.
Sharding, replication, caching. Each removes a bottleneck and hands you a caveat. Name both.
| Move | What it is | Buys you | Costs you | Caveat to name |
|---|---|---|---|---|
| Sharding | split different rows across machines | write & storage headroom | cross-shard joins & transactions get hard | the shard key is forever |
| Replication | keep the same data on several machines | read scale or staying up | copies lag the leader | reads can be stale |
| Caching | a fast copy of an answer you already computed | huge read relief, low latency | invalidation & stampedes | data can be stale |
Sharding splits different data across machines (for write/size scale). Replication copies the same data onto several machines (for read scale or availability). Most real systems do both.
The real products, and the lever you actually pull. Each card: reads how to scale reads, writes how to scale writes, lever the main mechanism, trade-off what it costs.
PgBouncer pooling. Scales reads almost linearly.ProxySQL, plus a cache.ONE, QUORUM). Reads scale with the cluster.Pattern across the table: reads almost always scale with replicas + a cache; writes scale with sharding (built-in for Mongo / Dynamo / Cassandra, bolt-on via Citus / Vitess for Postgres / MySQL). Redis and the derived stores (Elastic, ClickHouse) are about choosing the right copy, not the source of truth.
Splitting rows across machines. The shard key is the one decision that matters, and you live with it for years.
| Strategy | Splits by | Range reads | Hot-spot risk | Rebalancing |
|---|---|---|---|---|
| Range | value bands (A–M, N–Z) | easy | high — new data clumps | split a band |
| Hash | hash(key) → bucket | lost | low — spreads evenly | hard — use a ring |
| Directory | a lookup table maps key → shard | flexible | you control it | flexible — extra hop |
user_id, tenant_id; bad: country, status, a date).WHERE, one shard answers; if not, every query fans out to all of them.id % NWith id % N, adding one machine moves ~75% of keys, a brutal migration. Consistent hashing (a ring, with virtual nodes) moves only a small slice when a node joins or leaves. This is how Cassandra and DynamoDB scale without downtime.
Copies of the same data, for read scale or for staying up. Two choices define it: wait for copies or not, and one writer or many.
| Choice | Option | Pro | Con | Use when |
|---|---|---|---|---|
| Sync? | Synchronous | no loss on failover | slower writes; one slow copy stalls all | money, one sync follower |
| Asynchronous | fast writes | can lose the last few seconds | default for most data | |
| Writers? | Leader-based | simple, ordered | failover gap, single writer | almost everything |
| Leaderless (quorum) | always available | you resolve write conflicts | Dynamo, Cassandra, multi-region |
Async copies are always a little behind. The classic symptom is read-your-writes: a user posts something, the next read hits a lagging copy, and it’s “gone.” Fix: read a user’s own recent writes from the leader, everything else from replicas. Always monitor the lag.
A fast copy of an answer, kept close. Layers (browser → CDN → app/Redis → DB); answer as early as you can. The pattern decides who fills it.
| Pattern | How it fills | Write speed | Main risk |
|---|---|---|---|
| Cache-aside | app checks cache; on miss reads DB and stores it | normal | stale until TTL; first read is slow |
| Write-through | every write updates cache and DB | slower writes | always fresh, but caches unread data |
| Write-behind | write to cache now, flush to DB later | fastest | data lost if cache dies before flush |
Two strategies, and most bugs come from mixing them. TTL (time-based): trust it for N seconds, simple, wrong for up to the TTL. Explicit (event-based): delete on change, fresh, but you must catch every write path. Don’t promise “fresh”, promise a named staleness window.
A hot key expires and a million requests miss at once, stampeding the DB. Fixes: jitter the TTL so keys don’t expire together; request coalescing so the first miss fetches and the rest wait; refresh-ahead so it rebuilds in the background before it expires.
What breaks first, the four traps, and the four-sentence move that keeps you honest.
What breaks first
The four traps
1. name the bottleneck (reads? writes?)
2. pick the move (replica / cache / shard)
3. name what it buys
4. own what it costs
“Writes won’t fit on one box, so I shard by tenant_id. That buys linear write headroom. It costs me: big tenants run hot, and cross-tenant reports get hard.” That is a staff answer. “I’d add Cassandra” is a reflex.
Climb the ladder, name the bill. Reads scale with replicas and caches; writes scale with sharding; every copy is a staleness window you must name. Vertical before horizontal, always.
Companion to S4 · Data at Scale: Sharding, Replication, Caching. Further reading: System Design Primer · MongoDB sharding · Redis Cluster scaling · DynamoDB partition keys.