On this page
Description
A cache is a small, fast copy of data you'd otherwise fetch from somewhere slower. Picture a product page read 20,000 times a second while the product changes a few times a day: every read runs the same database query, against a database built for perhaps 3,000 queries a second. Keep a copy in memory and most of those reads never reach the database.
Adding Redis is easy and works on day one. Then the complaints start: a price change doesn't show up for an hour; a flash sale expires a hot key and the database falls over in the thirty seconds it takes to refill; the Redis node goes down and takes the database with it. Caching is easy to add and easy to get wrong, and the questions are always the same. In this lesson:
- Where caches sit - from the browser to the database.
- Getting data in (cache-aside and read-through) and handling writes.
- Getting stale data out - and why you delete rather than update.
- Eviction: what to throw away when memory fills up.
- Stampedes, hot keys, and what happens when the cache itself fails.
- CDNs, and which pattern fits which situation.
Where caches sit
Each layer answers what it can and passes the rest inward; the further in a request gets, the slower and more expensive it is.
- In the client - browser caches, controlled by
Cache-Controlheaders. - At the edge - a CDN caching static files, images and video close to users.
- In the app server's memory - fastest, but each server has its own copy, so invalidation is per server.
- A shared cache tier - Redis or Memcached, one copy shared by all app servers. This is what "add a cache" usually means.
Getting data in: read patterns
Cache-aside (lazy loading) is the default:
value = cache.get(key)
if value is None:
value = db.query(key)
cache.set(key, value, ttl=300)
return value
Only data someone actually asks for gets cached, and a cache failure degrades to database reads. Read-through is the same idea with the cache (or a library in front of it) doing the database load, so the app only talks to the cache.
Writes
| Pattern | On write | Upside | Downside |
|---|---|---|---|
| Write-around (with cache-aside) | Write the database; delete or skip the cache | Simple; the cache holds only what's read | The next read is a miss |
| Write-through | Write the cache and the database together | Cache always fresh | Every write pays both; caches data nobody reads |
| Write-back (write-behind) | Write the cache; flush to the database later | Fastest writes; batches them | Writes still only in the cache are lost if it fails |
Write-back suits counters and metrics you can afford to lose a few seconds of - not orders or payments.
Getting stale data out
Delete, don't update. On a change, write the database and then delete the cache key; the next read loads the fresh value. Updating the key in place (cache.set(key, newValue)) races: two writers can commit to the database in one order and set the cache in the other, leaving the older value cached indefinitely:
Even delete-on-write has a rare race - a reader that loaded the old value just before the write can set it just after the delete. A TTL bounds how long any such mistake lives: pick one the data can tolerate (seconds for prices, hours for a blog post). And some reads shouldn't come from a long-lived cache at all: an account balance before a payment, stock at checkout, anything a decision depends on.
Eviction
Memory fills up. LRU (least recently used) evicts what hasn't been read for longest - good when recent popularity predicts future reads, as with trending content. LFU (least frequently used) favors keys with many hits over time. FIFO ignores popularity and evicts hot items just because they're old. A cache with no eviction simply fails writes (or runs out of memory) when full.
Stampedes and hot keys
Stampede (thundering herd). A popular key expires; in the next 50 ms, 5,000 requests miss and all query the database for the same row. Fixes:
- Request coalescing - the first miss takes a short lock (or a single-flight promise) and reloads; the others wait for its result instead of querying.
- Early refresh - reload a key shortly before it expires.
- Jittered TTLs - 300 s ± 10%, so keys filled together don't expire together.
Hot keys. One key - a celebrity's profile - gets 500,000 reads a second, more than one cache node can serve. Consistent hashing spreads keys across nodes; it can't split one key. Replicate the hot key instead: store copies under profile:42#1..#8 and read a random one, or keep a short-lived (a few seconds) copy in each app server's memory.
When the cache fails
At a 90% hit rate the database sees only 10% of reads. If the cache disappears, the database suddenly gets all of them - ten times its normal read load - often exactly when you can least afford it. Plan for it: run the cache replicated across zones, keep the app able to reach the database without it (slower but correct), shed or rate-limit load while the cache warms back up, and don't size the database only for misses if an outage must stay survivable.
CDNs
A CDN is a cache on the edge of the network, close to users. Static assets, images and video segments are cached at hundreds of locations; Cache-Control headers and versioned file names (app.3f9a.js) control freshness. For media-heavy products it removes most bandwidth from your servers.
Which pattern?
| Situation | Pattern |
|---|---|
| Read-heavy data that changes occasionally (products, profiles) | Cache-aside, delete on write, TTL as a safety net |
| Data that must be fresh the moment it's written, read right after | Write-through (or skip the cache for that read) |
| High-rate counters, metrics, view counts | Write-back, accepting a few seconds of loss |
| Decisions on money or stock (balance before a payment) | Read the database, not the cache |
| Static files, images, video | CDN with versioned file names |
| One key with enormous traffic | Replicated hot key, or a few seconds in app memory |
Going further. The deep dive (next tab) quantifies hit ratios and what the last percent is worth, simulates LRU on skewed traffic, shows how Facebook closed the cache race at scale (leases and invalidation from the database's commit log), and covers eviction algorithms, Redis failover, cache penetration and avalanches, private data in shared caches, and HTTP's stale-while-revalidate.
Check yourself
Try to answer before opening each one - recalling beats re-reading.
1. Why delete the cache key on a write rather than setting the new value?
Show answer
Two concurrent writers can update the database in one order and the cache in the other, leaving the older value cached until it expires. Deleting makes the next reader load whatever the database holds now.
2. A cache has a 95% hit rate. It fails. What happens to database read load?
Show answer
It goes from 5% of reads to 100% - twenty times higher. A database sized for misses won't survive that without load shedding or a replicated cache.
3. When is write-back the wrong choice?
Show answer
When losing the most recent writes is unacceptable - orders, payments, anything durable - because writes that haven't been flushed are lost if the cache node fails.
Key takeaways
- Cache-aside is the default: read the cache, load on a miss, fill the cache.
- Write-through is fresh but slow to write; write-back is fast but can lose writes.
- On writes: update the database, then delete the key; a TTL bounds staleness.
- Evict with LRU (or LFU); FIFO evicts popular items.
- Coalesce misses and jitter TTLs against stampedes; replicate hot keys.
- A cache failure multiplies database load by 1 ÷ (miss rate) - plan for it.
- CDNs cache static content and media at the edge.