Reference · gazar.dev ← All cheat sheets

Data at scale · caching

The caching strategies cheat sheet.

S4 said it: caching is the first lever you pull to scale reads. This is the whole toolkit on one page: the read strategies and write strategies with sequence diagrams, how they pair, the consistency you actually get, eviction and invalidation, the failure modes that bite (stampede, hot key, penetration, avalanche), the layers from browser to CDN to Redis, which technology to use (Redis vs Memcached), the architecture (local, distributed, near-cache, sharding), and how to size and measure a cache. Companion to S4 · Data at Scale.

The one rule behind everything here

A cache trades freshness for speed and load. Every cache is a faster copy that can be stale, so the whole craft is three decisions: how stale the data can be, where the copy lives, and how it is written and thrown away. Get those right and reads get an order of magnitude cheaper; get the invalidation wrong and the cache stops being a speedup and becomes a bug.

First: a cache is a faster copy that can lie

Before any strategy, hold the trade. A cache puts a copy of data closer or faster than the source of truth. That buys low latency and shields the origin from load. The price is that the copy can drift, so you are always choosing how much staleness is acceptable, and a cache only pays off when the same data is read many times before it changes.

What you gain

Latency: a memory or edge read (sub-ms) instead of a database or origin hit (tens of ms). Load shielding: most reads never reach the source, so it scales, and a high hit ratio is a large multiplier on origin capacity.

What you pay

Staleness: the copy can be out of date. Complexity: invalidation, consistency, and a new component that can itself fail. "There are only two hard things in computer science: cache invalidation and naming things."

The two numbers that justify a cache

effective read latency ≈ hit_ratio × cache_latency
                       + (1 − hit_ratio) × miss_latency

  hit_ratio 0.95, cache 1ms, miss 50ms
    = 0.95×1 + 0.05×50 = 0.95 + 2.5 = 3.45ms   (vs 50ms uncached, ~14× faster)

load reaching the origin ≈ (1 − hit_ratio) × total_reqs

  95% hit ⇒ origin sees 5% of traffic  ....... a 20× read shield
  99% hit ⇒ origin sees 1% of traffic  ....... a 100× read shield

The hit ratio is the whole game. Latency and load both improve non-linearly as it climbs, which is why the last few percent (stampede protection, right-sized TTLs, good keys) are worth chasing.

The trap

Caching data that must always be correct without a plan for staleness. A cache in front of a balance or an inventory count, with a careless TTL, serves wrong numbers. Decide the tolerable staleness per item first: some data caches for a day, some for a second, some not at all. Staleness budget is an input to the design, not an accident of the default TTL.

The strategy landscape

Caching strategies split on two axes: how a read is served (and who loads on a miss) and how a write reaches the cache and the store. Pick one from each; most systems land on cache-aside reads with a TTL and either write-around or delete-on-write.

Axis	Strategy	In one line	Default?
Read	Cache-aside	app checks cache, loads from DB on a miss, then populates it (lazy)	yes
Read	Read-through	app talks only to the cache; the cache loads from DB on a miss	library-dependent
Write	Write-through	write cache and DB synchronously, together, so the cache is always fresh	when freshness > write latency
	Write-back	write to cache now, ack, flush to DB asynchronously later	only for write bursts
	Write-around	write straight to DB, skip (or delete) the cache; next read repopulates	common with cache-aside

The one-line map

Reads: cache-aside is the default (lazy, simple, resilient); read-through moves the loading into the cache when your library supports it. Writes: write-around + delete-on-write is the common partner for cache-aside; write-through when you cannot tolerate a stale cache and accept slower writes; write-back only when write throughput beats durability. "Cache-aside + TTL" covers most systems you will ever build.

Read strategies: cache-aside vs read-through

Both serve a hit from the cache and load the source on a miss. The only difference is who owns the loading logic: your application code (aside) or the cache layer itself (through). Each card: model how it works, use when to reach for it, watch the catch.

Cache-asidelazy loading

Model: the app orchestrates: check cache; on a miss read the DB, then write the value back into the cache with a TTL.
Use: the default for read-heavy data. Simple, and only ever caches what is actually requested. Survives a cache outage (reads just fall through to the DB).
Watch: the first read of any key is a miss (cold cache). Loading logic is duplicated at every call site, so it can drift.

Read-throughcache loads on miss

Model: the app asks only the cache; the cache (or its client library) fetches from the DB on a miss and stores it, transparently.
Use: when you want one place for loading logic and a uniform API. Common with cache libraries and some managed caches.
Watch: needs a cache that supports a loader. A cache outage now blocks reads unless you add a fallback. Same cold-miss on first read.

Cache-aside: hit vs miss

sequenceDiagram
  autonumber
  participant App
  participant Cache
  participant DB
  rect rgb(214,245,227)
  Note over App,Cache: cache HIT, the fast path
  App->>Cache: get(k)
  Cache-->>App: value
  end
  rect rgb(254,228,226)
  Note over App,DB: cache MISS
  App->>Cache: get(k)
  Cache-->>App: nil
  App->>DB: read(k)
  DB-->>App: row
  App->>Cache: set(k, row, ttl)
  end

Read-through: loading lives in the cache

sequenceDiagram
  autonumber
  participant App
  participant Cache
  participant DB
  App->>Cache: get(k)
  Note over Cache: miss, the cache loads
  Cache->>DB: read(k)
  DB-->>Cache: row
  Note over Cache: store k = row
  Cache-->>App: row
  Note over App,DB: the App never touches the DB

Write strategies: through, back & around

Three ways a write reaches the cache and the store. They trade write latency, freshness, and durability against each other. The right choice depends on whether just-written data is read soon, and how much a lost write costs.

Write-throughsync, always fresh

Model: write to cache and DB synchronously in the same operation; both are updated before you ack.
Use: when a stale cache is unacceptable and the data is read soon after write. The cache is never behind the DB.
Watch: every write pays two hops, so writes are slower. Caches data that may never be read (write-heavy, read-rarely wastes memory).

Write-backwrite-behind, async

Model: write to the cache, ack immediately, then flush to the DB asynchronously (often batched / coalesced).
Use: very high write throughput or bursty writes; batching many updates into fewer DB writes (counters, metrics, buffers).
Watch: if the cache node dies before the flush, those writes are lost. Needs replication/persistence to be safe. Complex ordering.

Write-aroundskip the cache

Model: write straight to the DB and delete (or skip) the cache entry; the next read misses and repopulates fresh.
Use: the natural partner to cache-aside. Best when written data is not read again soon, so you avoid caching cold writes.
Watch: the read right after a write is a guaranteed miss (higher read latency on recently-written keys).

Write-through: cache in the write path

sequenceDiagram
  autonumber
  participant App
  participant Cache
  participant DB
  App->>Cache: write(k, v)
  Cache->>DB: write(k, v)
  DB-->>Cache: ok
  Cache-->>App: ok
  Note over App,DB: both updated before ack, never stale

Write-back: ack now, flush later

sequenceDiagram
  autonumber
  participant App
  participant Cache
  participant DB
  App->>Cache: write(k, v)
  Cache-->>App: ack, fast
  Cache->>DB: async batched flush, coalesces N writes
  Note over Cache,DB: risk: cache dies before flush, writes lost

Write-around: DB first, invalidate cache

sequenceDiagram
  autonumber
  participant App
  participant Cache
  participant DB
  App->>DB: write(k, v)
  App->>Cache: delete(k)
  Note over App,Cache: next read misses, loads fresh from DB

Delete, don't update, on write

prefer:  write DB  →  delete cache key
avoid:   write DB  →  set cache = new value

why: two concurrent writers that "set" can
interleave and leave the OLD value cached.
a delete lets the next read repopulate the
truth, and is naturally idempotent.

The write ordering trap

With cache-aside, update the DB first, then delete the cache key (not "set" it). Two writers that each set the cache can interleave so the cache ends up holding the older value indefinitely. Deleting sidesteps the race, and is idempotent. The residual risk (a read repopulating stale between your DB write and the delete) is bounded by the TTL, and eliminated by delaying/duplicating the delete if you need it airtight.

Read × write pairings

Read and write strategies combine. These are the pairings you actually see in production, and what each is good for.

Read + write	Behaviour	Reach for it when
Cache-aside + write-around + TTL	lazy load on read; writes go to DB and drop the key; TTL bounds staleness	the default for read-heavy data. Simple, resilient, memory-efficient
Cache-aside + delete-on-write	as above but delete the key on every write for tighter freshness	you can catch all write paths and want fresher-than-TTL reads
Read-through + write-through	cache owns both load and write; always fresh, uniform API	a managed cache library; freshness matters and writes are read soon
Read-through + write-back	cache absorbs reads and writes; DB updated async	write-heavy, latency-critical, loss window acceptable (counters, metrics)

How to pick in one breath

Start at cache-aside + TTL. Add delete-on-write if the TTL is too loose for the data. Move to write-through only if a stale cache is genuinely unacceptable and you will pay slower writes. Reach for write-back only when write throughput is the binding constraint and you have made the loss window safe. Everything else is tuning.

Consistency & the stale window

A cache makes your system eventually consistent by construction: for some window, the copy can disagree with the truth. The strategy decides how big that window is and who might notice.

Approach	Stale window	Cost
TTL only	up to the TTL after a change	trivial; tune the TTL to the tolerable staleness
TTL + delete-on-write	near-zero on paths you catch; TTL elsewhere	must hit every write path; a missed one falls back to TTL
Write-through	zero (cache and DB move together)	slower writes; only for keys you write through
Event-driven invalidation	small (propagation delay of the event)	a bus/CDC pipeline to publish changes; more moving parts

Read-your-own-writes

The staleness users hate most is not seeing their own change. Fix it locally: after a user's write, serve their next read from the source (or from the value you just wrote), while everyone else rides the cache. You bound the visible-staleness problem to exactly the person who would notice, without giving up the cache for everyone. This is the caching face of the read-your-writes guarantee.

Eviction: a cache is finite

A cache has bounded memory, so when it fills, something must go. The policy decides what. TTL is a special case: it evicts by age, and doubles as your simplest invalidation.

Policy	Evicts	Good when
LRU	least recently used	the default; recency predicts reuse well
LFU	least frequently used	stable hot set; a viral spike should not evict steady favourites
FIFO	oldest inserted	simple; ignores access, usually worse than LRU
Random	a random entry	cheap, no bookkeeping; surprisingly OK, used as a Redis option
TTL	anything past its expiry	bounds staleness; layers on top of any policy above

TTL is doing two jobs, and Redis picks a policy for you

A TTL both frees memory and, more importantly, bounds staleness: it is the simplest invalidation there is. Set it to the longest staleness the data can tolerate. Combine size control (LRU/LFU) with a TTL (freshness control): Redis exposes this as maxmemory-policy, where allkeys-lru and volatile-ttl are common choices. Pick allkeys-* to treat the whole store as a cache, volatile-* to only evict keys that carry a TTL.

Invalidation: the hard half

Keeping the cache from serving stale data after the source changes. There is no perfect answer, only a trade between freshness, complexity, and load.

TTL expirytime-based

How: let entries expire after N seconds; the next read repopulates.
Watch: simple and robust, but data is stale for up to the TTL. Tune per item.

Delete / update on writeevent-based

How: delete the cache entry the moment the DB changes (prefer delete over update, see §4).
Watch: fresher, but you must catch every write path; a missed one serves stale until the TTL saves you.

Versioned keysnever invalidate

How: put a version/hash in the key (user:42:v7, app.4f3a.js); a change is a new key.
Watch: sidesteps invalidation entirely; the old entry just ages out. Great for static assets and derived data.

The senior move: prefer expiry or versioning over manual deletes

Explicitly deleting cache entries on every write is where bugs live: miss one code path and you serve stale data indefinitely. A TTL bounds the damage automatically; versioned keys remove the problem by making a change a new key. Reach for targeted invalidation only when you truly cannot tolerate even a short TTL, and even then, keep a TTL underneath as a safety net.

Cache failure modes

The ways a cache turns from a speedup into an outage. Each has a known fix, and almost all of them are one shape: a synchronized miss slamming the origin.

Failure	What happens	Fix
Stampede	a popular key expires; thousands of concurrent misses all hit the DB at once (a.k.a. dog-piling)	single-flight so one request repopulates; or stale-while-revalidate
Hot key	one key gets so much traffic it saturates a single cache node	replicate the key across nodes; add a small local (L1) cache in front
Penetration	queries for keys that do not exist bypass the cache and hammer the DB	cache the negative result (short TTL); a Bloom filter of existing keys
Avalanche	many keys expire at the same instant; a load spike to the origin	jitter the TTLs so expiries spread out; pre-warm before expiry

Stampede & the single-flight fix

flowchart TB
  subgraph without["Without protection"]
    E1[key k expires] --> R1[1000 concurrent misses]
    R1 --> D1[(Database)]
    D1 --> M1[DB melts]
  end
  subgraph single["With single-flight, a per-key lock"]
    E2[key k expires] --> L1[1 request takes the lock]
    L1 --> D2[(Database)]
    D2 --> S2[repopulate cache]
    E2 --> W2[999 requests wait or serve stale]
    W2 --> S2
  end
  M1 ~~~ E2
  class M1 bad
  class S2 good
  classDef bad fill:#FEE4E2,stroke:#C8102E,color:#09244B
  classDef good fill:#D6F5E3,stroke:#1B7F46,color:#09244B

Stale-while-revalidate

flowchart TB
  A[read hits an expired entry] --> B[serve the STALE value now]
  A --> C[kick off ONE async refresh]
  C --> D[cache updated in background]
  D --> E[next reads get the fresh value]
  class B good
  classDef good fill:#D6F5E3,stroke:#1B7F46,color:#09244B

The pattern behind the fixes

Almost every cache failure is a synchronized miss hitting the origin. The fixes all spread or absorb that: single-flight so one request refills (not thousands), jittered TTLs so expiries do not align, stale-while-revalidate so nobody waits, and negative caching so non-existent keys do not slip through. This is the S9 thundering-herd idea, applied to caches.

The caching layers

Caching is not one box, it is a stack. A read can be served at any layer; the closer to the user, the cheaper and faster. Many systems run an L1 (in-process) in front of an L2 (shared Redis) to kill hot keys.

A read travels down until a layer has the answer

flowchart TB
  U([request]) --> BR["Browser cache
in the user's device"]
  BR -- miss --> CDN["CDN edge
near the user, global"]
  CDN -- miss --> L1["L1 in-process
local heap, kills hot keys"]
  L1 -- miss --> L2["L2 distributed
Redis / Memcached"]
  L2 -- miss --> DBC["Database cache
buffer pool, pages in RAM"]
  DBC -- miss --> DB[("Database
the source of truth")]
  class DB origin
  classDef origin fill:#FEE4E2,stroke:#C8102E,color:#09244B

Cache as close to the user as the data's freshness allows. Each layer up is faster and cheaper but harder to invalidate (you control Redis; you do not control the user's browser). That asymmetry is why versioned URLs matter more the higher you cache.

What to remember

A static asset is cached in the browser and the CDN and never touches your servers. A hot row is cached in Redis, and if one key is scorching, a tiny L1 in each app instance absorbs it with no network hop. A session lives in Redis too. The design goal is to push each piece of data to the highest layer its staleness budget permits.

CDN: caching at the edge

A CDN is a global cache layer in front of your origin, serving content from a location near each user. It is the same caching idea pushed all the way out to the edge, and it is how you serve heavy or popular content cheaply and fast worldwide.

Pull vs pushhow content arrives

Pull: edge fetches from origin on the first miss, then caches it. The default; lazy and self-managing (cache-aside at the edge).
Push: you upload content to the CDN ahead of time. For large files or known launches you want pre-warmed.
Use: pull for almost everything; push to pre-warm a big release.

Cache-Controlthe headers

Knobs: max-age (how long), public/private (CDN may cache or not), no-store (never), stale-while-revalidate, ETag for revalidation.
Pattern: immutable, versioned assets (app.4f3a.js) get a year-long max-age; HTML gets a short one or revalidation.
Watch: caching a personalized response as public leaks one user's data to another. Mark it private.

The immutable-asset trick

Put a content hash in the filename and you never invalidate: a change produces a new URL, so the CDN can cache the old one forever and the browser fetches the new one automatically. This is why build tools fingerprint assets. Cache HTML briefly, fingerprint everything it links, and the edge serves almost everything, the same versioned-key idea from §8, at the outermost layer.

Cache technologies: Redis vs Memcached & friends

A strategy needs a store. For the shared (L2) tier the real choice is almost always Redis vs Memcached; around them sits a family of caches for other layers. Pick by what you need beyond a key→value GET/SET.

Dimension	Redis	Memcached
Data model	rich: strings, hashes, lists, sets, sorted sets, streams, bitmaps, HLL, geo, pub/sub	simple: key → opaque blob only
Threading	command path single-threaded (I/O threads in 6+); predictable, atomic	multi-threaded; scales raw GET/SET across cores
Persistence	RDB snapshots + AOF log; restarts warm	none; purely volatile
Replication / HA	replicas + Sentinel failover + Cluster (sharding)	none native; sharding is client-side
Atomicity / extras	atomic ops, transactions, Lua, pub/sub, TTL per key	CAS + counters; deliberately minimal
Value / memory	values up to 512MB; more per-key overhead	1MB item default; very low overhead, slab allocator

Reach for Memcached when

a simple, huge, ephemeral string/blob cache is all you need
you want multi-core throughput on plain GET/SET
you value operational simplicity and low memory overhead

Reach for Redis when

you need data structures, atomic ops, or pub/sub
you need persistence, replication or HA (sessions that survive restarts)
the cache doubles as a rate limiter, leaderboard, queue or lock

Layer	Typical technologies	Role
In-process (L1)	Caffeine / Guava (JVM), `lru-cache` (Node), `functools.lru_cache` (Py), Ehcache	fastest, per-instance, no network
Distributed (L2)	Redis, Memcached, Valkey, Hazelcast, Apache Ignite	one shared tier all app nodes use
Reverse-proxy / HTTP	Varnish, nginx `proxy_cache`, Apache Traffic Server	cache whole responses in front of the app
Edge / CDN	Cloudflare, Fastly, CloudFront, Akamai	cache near the user, globally (§11)
DB-integrated	DynamoDB DAX, ProxySQL, buffer pool, materialized views	cache in or beside the database

The default, and when to leave it

Default to Redis for the shared tier: it does everything Memcached does and far more, and the single-threaded command path makes its atomic operations easy to reason about. Choose Memcached only when you specifically want a dead-simple, multi-threaded, volatile blob cache and none of the extras. For the fastest possible reads on a tiny hot set, neither, use an in-process cache and skip the network entirely.

Cache architecture & topology

The same store can be wired in different shapes. The topology decides where the copy lives, how many copies exist, and what happens as you add nodes. This is the "what type of cache" question at the architecture level.

Topology	Where it lives	Wins	Costs
Local / in-process (L1)	inside each app instance's heap	fastest; no network, no serialization	a copy per node ⇒ inconsistent, N× memory, cold on deploy
Distributed / remote (L2)	a separate shared cache tier	one consistent copy; scales alone; survives app restarts	a network hop + serialization; a dependency that can fail
Near-cache (L1 + L2)	small local L1 over shared L2	hot keys served locally, bulk shared	L1 invalidation is hard (pub/sub or short TTL)
Reverse-proxy / HTTP	in front of the app	caches whole responses; offloads the app entirely	coarse-grained; keyed per URL/headers

Three topologies, side by side

flowchart LR
  subgraph local["Local L1: fast, per-node, inconsistent"]
    a1["app 1 + cache"]
    a2["app 2 + cache"]
    a3["app 3 + cache"]
  end
  subgraph dist["Distributed L2: one shared copy, +1 hop"]
    b1[app 1] --> R1[(Redis)]
    b2[app 2] --> R1
    b3[app 3] --> R1
  end
  subgraph near["Near-cache L1 + L2: hot local, bulk shared"]
    c1["app 1 L1"] --> R2[(Redis)]
    c2["app 2 L1"] --> R2
    c3["app 3 L1"] --> R2
  end

Scaling the distributed tier: shard, hash, replicate

One cache node is a capacity and failure ceiling. You scale out with three moves: sharding (partition keys across nodes so each holds a slice), consistent hashing (so adding or losing a node reshuffles ~1/N of keys instead of nearly all), and replication (replicas for HA and read fan-out). Redis Cluster does all three; with Memcached the client library shards.

Consistent hashing: each key maps to the next node clockwise

flowchart LR
  nA(("node A")) --> nB(("node B"))
  nB --> nC(("node C"))
  nC --> nD(("node D"))
  nD --> nA
  k3["key 3"] -.-> nA
  k1["key 1"] -.-> nB
  k2["key 2"] -.-> nC
  k4["key 4"] -.-> nD
  classDef key fill:#DCEBFE,stroke:#4A90D9,color:#09244B
  class k1,k2,k3,k4 key

Add or remove a node and only the keys between it and its neighbour move (~1/N), not the whole keyspace. With naive hash(key) mod N, changing N remaps almost every key, a mass miss storm against the origin. Virtual nodes (many points per physical node) even out the load.

What to remember

Start with a distributed cache (shared Redis) so every node sees one copy. Add an in-process L1 only when a hot key or a chatty path needs it, and accept that L1 is eventually consistent (bound it with a short TTL or invalidate over pub/sub). When you outgrow one cache node, shard with consistent hashing and add replicas, don't just make the node bigger.

Keys, sizing & metrics

The strategy is only half of it. A cache lives or dies on its key design, its TTLs, and whether you are actually measuring the hit ratio.

Key design

namespace + entity + id: user:42:profile
include every input that changes the value (locale, version, feature flags)
hash long/compound keys; keep them short and deterministic
version in the key to sidestep invalidation

Sizing the TTL

TTL = the longest staleness the data can tolerate
longer TTL ⇒ higher hit ratio but staler data
add jitter so expiries do not align (anti-avalanche)
size memory to hold the working set, not everything

What to measure

hit ratio (the headline number)
p50/p99 latency, hit vs miss
eviction rate + memory used / max
origin load (should be ~1−hit_ratio of traffic)

The number that tells you if it is working

The hit ratio is the cache's report card. A low ratio means the TTL is too short, the working set does not fit (constant eviction), or the keys are too fine-grained to be reused. A high ratio on data that is quietly stale is worse: measure freshness too. If you cannot see the hit ratio and the eviction rate, you are flying blind, and a cache you cannot observe is a cache you cannot trust.

What do I cache, and where?

Match the data to the highest layer its freshness budget allows, and the strategy to its write needs.

What is the data? Match it to a home

flowchart LR
  Q{What is the data?}
  Q --> S["static asset js/css/img"] --> S2["CDN + browser, immutable key, max-age 1y"]
  Q --> P["public slow-changing page"] --> P2["CDN short TTL or stale-while-revalidate"]
  Q --> H["hot DB row or computed"] --> H2["Redis, cache-aside + TTL, L1 if hot"]
  Q --> SE["user session"] --> SE2["Redis or signed cookie"]
  Q --> PR["personalized response"] --> PR2["private cache only, never shared CDN"]
  Q --> WC["write-heavy counter"] --> WC2["write-back, batched flush"]
  Q --> EX["must-be-exact balance or stock"] --> EX2["do not cache, or write-through tiny TTL"]
  classDef ans fill:#DCEBFE,stroke:#4A90D9,color:#09244B
  class S2,P2,H2,SE2,PR2,WC2,EX2 ans
  classDef warn fill:#FEE4E2,stroke:#C8102E,color:#09244B
  class EX2 warn

You want…	Reach for	Notes
Read-heavy DB data, simple	Cache-aside + TTL (Redis)	the default
Fresher than a TTL allows	Cache-aside + delete-on-write	catch every write path
Always-fresh cache	Write-through	slower writes
Absorb write bursts	Write-back	risk a loss window
Global static delivery	CDN + immutable assets	fingerprint filenames
Tame a single hot key	L1 in-process + replicate	local cache absorbs it
Stop a stampede	Single-flight + jittered TTL	one refill, spread expiry
Stop bogus-key floods	Negative cache / Bloom filter	cache the miss

Interview questions, with crisp answers

The caching questions that come up, and the sentence or two that shows judgement rather than recall.

Q · Cache-aside vs read-through?

Both serve hits from cache and load on a miss; the difference is who owns loading. Cache-aside puts it in the app (simple, survives a cache outage). Read-through puts it in the cache library (one place, uniform API, but a cache outage blocks reads). Default to cache-aside.

Q · Write-through vs write-back vs write-around?

Write-through writes cache+DB together (always fresh, slower writes). Write-back acks from cache and flushes later (fast, risks a loss window). Write-around skips the cache and lets the next read repopulate (best when writes aren't read soon).

Q · On a write, update or delete the cache?

Delete. Two writers that each "set" the cache can interleave and leave the old value cached. Deleting lets the next read repopulate the truth and is idempotent. Update DB first, then delete the key.

Q · How do you invalidate a cache?

Prefer a TTL (bounds staleness automatically) or versioned keys (a change is a new key). Delete-on-write is fresher but fragile: miss one write path and you serve stale forever. Keep a TTL underneath as a safety net.

Q · What is a cache stampede, and the fix?

A hot key expires and thousands of concurrent misses hit the DB at once. Fix with single-flight (one request repopulates while others wait) or stale-while-revalidate (serve stale, refresh once in the background), plus jittered TTLs so expiries don't align.

Q · LRU vs LFU?

LRU evicts the least recently used, recency predicts reuse and it is the default. LFU evicts the least frequently used, better when a stable hot set should survive a brief traffic spike. Most systems use LRU with a TTL.

Q · How does a CDN avoid invalidation?

Immutable, content-hashed filenames: a change is a new URL, so the edge caches the old one forever and the browser fetches the new automatically. Cache HTML briefly, fingerprint everything it references.

Q · When should you NOT cache?

When the data must be exact and current (a balance, live inventory) and even a second of staleness is wrong, or when reads are rare so the cache never pays off. Caching the wrong thing turns a speedup into a correctness bug.

Q · Redis or Memcached?

Memcached for a dead-simple, multi-threaded, purely-volatile blob cache at high throughput. Redis when you need data structures, persistence, replication/HA, or atomic ops and pub/sub (sessions, rate limiters, leaderboards). Default to Redis, it does everything Memcached does and more.

Q · Local (in-process) vs distributed cache?

Local is fastest (no network) but each node holds its own copy, so it's inconsistent and costs N× memory. Distributed (Redis) is one shared copy that survives restarts and scales alone, at a network hop. Many systems run a small local L1 in front of a shared L2 to absorb hot keys.

Q · Why consistent hashing for a cache cluster?

With hash(key) % N, changing N remaps almost every key, so adding a node triggers a mass miss storm against the origin. Consistent hashing moves only ~1/N of keys when a node joins or leaves; virtual nodes even out the load.

Traps & the repeatable move

The ways caching goes wrong, and the habit that catches them.

The traps

No TTL, stale data lives forever and you debug a ghost.
Set-on-write instead of delete, a race leaves the old value cached.
Manual invalidation everywhere, one missed write path serves stale indefinitely.
Synchronized TTLs, an avalanche of expiries spikes the origin.
No stampede protection, a hot key's expiry melts the database.
Caching personalized data publicly, one user sees another's response.
Caching must-be-exact data, a stale balance or oversold inventory.
No hit-ratio metric, a cache you can't see is a cache you can't trust.

The repeatable move

1. how stale can this item be? sets the TTL
2. which layer? browser / CDN / L1 / Redis / DB
3. which read + write? aside+TTL (default), through, back
4. on write, delete the key (DB first)
5. protect the miss: single-flight + jitter + negative cache

The tell in a design review: a cache with no TTL, a set-on-write race, or a personalized response marked public. Name the staleness budget per item, and push back on caching things that must be exact.

The one-liner to remember

Cache as close to the user as the freshness budget allows, bound staleness with a TTL or a versioned key, delete-don't-update on write, and protect the miss. This whole sheet is the S4 "cache to scale reads" slide unpacked, and the CDN is just this same idea pushed to the edge.

Companion to S4 · Data at Scale. Further reading: AWS · caching best practices · Redis · eviction policies · MDN · HTTP caching · Cloudflare · what is a CDN.