Caching

---
name: caching
model: standard
description: Caching strategies, invalidation, eviction policies, HTTP caching, distributed caching, and anti-patterns. Use when designing cache layers, choosing eviction policies, debugging stale data, or optimizing read-heavy workloads.
---

# Caching Patterns

> A well-placed cache is the cheapest way to buy speed. A misplaced cache is the most expensive way to buy bugs.

## Cache Strategies

| Strategy | How It Works | When to Use |
|----------|-------------|-------------|
| **Cache-Aside (Lazy)** | App checks cache → miss → reads DB → writes to cache | **Default choice** — general purpose |
| **Read-Through** | Cache fetches from DB on miss automatically | ORM-integrated caching, CDN origin fetch |
| **Write-Through** | Writes go to cache AND DB synchronously | Read-heavy with strong consistency |
| **Write-Behind** | Writes go to cache, async flush to DB | High write throughput, eventual consistency OK |
| **Refresh-Ahead** | Cache proactively refreshes before expiry | Predictable access patterns, low-latency critical |

```
Cache-Aside Flow:

  App ──► Cache ──► HIT? ──► Return data
              │
              ▼ MISS
          Read DB ──► Store in Cache ──► Return data
```

---

## Cache Invalidation

| Method | Consistency | When to Use |
|--------|-------------|-------------|
| **TTL-based** | Eventual (up to TTL) | Simple data, acceptable staleness |
| **Event-based** | Strong (near real-time) | Inventory, profile updates |
| **Version-based** | Strong | Static assets, API responses, config |
| **Tag-based** | Strong | CMS content, category-based purging |

### TTL Guidelines

| Data Type | TTL | Rationale |
|-----------|-----|-----------|
| Static assets (CSS/JS/images) | 1 year + cache-busting hash | Immutable by filename |
| API config / feature flags | 30–60 seconds | Fast propagation needed |
| User profile data | 5–15 minutes | Tolerable staleness |
| Product catalog | 1–5 minutes | Balance freshness vs load |
| Session data | Match session timeout | Security requirement |

---

## HTTP Caching

### Cache-Control Directives

| Directive | Meaning |
|-----------|---------|
| `max-age=N` | Cache for N seconds |
| `s-maxage=N` | CDN/shared cache max age (overrides max-age) |
| `no-cache` | Must revalidate before using cached copy |
| `no-store` | Never cache anywhere |
| `must-revalidate` | Once stale, must revalidate |
| `private` | Only browser can cache, not CDN |
| `public` | Any cache can store |
| `immutable` | Content will never change (within max-age) |
| `stale-while-revalidate=N` | Serve stale for N seconds while fetching fresh |

### Common Recipes

```
# Immutable static assets (hashed filenames)
Cache-Control: public, max-age=31536000, immutable

# API response, CDN-cached, background refresh
Cache-Control: public, s-maxage=60, stale-while-revalidate=300

# Personalized data, browser-only
Cache-Control: private, max-age=0, must-revalidate
ETag: "abc123"

# Never cache (auth tokens, sensitive data)
Cache-Control: no-store
```

### Conditional Requests

| Mechanism | Request Header | Response Header | How It Works |
|-----------|---------------|-----------------|-------------|
| **ETag** | `If-None-Match: "abc"` | `ETag: "abc"` | Hash-based — 304 if match |
| **Last-Modified** | `If-Modified-Since: <date>` | `Last-Modified: <date>` | Date-based — 304 if unchanged |

Prefer ETag over Last-Modified — ETags detect content changes regardless of timestamp granularity.

---

## Application Caching

| Solution | Speed | Shared Across Processes | When to Use |
|----------|-------|------------------------|-------------|
| **In-memory LRU** | Fastest | No | Single-process, bounded memory, hot data |
| **Redis** | Sub-ms (network) | Yes | **Production default** — TTL, pub/sub, persistence |
| **Memcached** | Sub-ms (network) | Yes | Simple key-value at extreme scale |
| **SQLite** | Fast (disk) | No | Embedded apps, edge caching |

### Redis vs Memcached

| Feature | Redis | Memcached |
|---------|-------|-----------|
| Data structures | Strings, hashes, lists, sets, sorted sets | Strings only |
| Persistence | AOF, RDB snapshots | None |
| Pub/Sub | Yes | No |
| Max value size | 512 MB | 1 MB |
| **Verdict** | **Default choice** | Pure cache at extreme scale |

---

## Distributed Caching

| Concern | Solution |
|---------|----------|
| **Partitioning** | Consistent hashing — minimal reshuffling on node changes |
| **Replication** | Primary-replica — writes to primary, reads from replicas |
| **Failover** | Redis Sentinel or Cluster auto-failover |

**Rule of thumb:** 3 primaries + 3 replicas minimum for production Redis Cluster.

---

## Cache Eviction Policies

| Policy | How It Works | When to Use |
|--------|-------------|-------------|
| **LRU** | Evicts least recently accessed | **Default** — general purpose |
| **LFU** | Evicts least frequently accessed | Skewed popularity distributions |
| **FIFO** | Evicts oldest entry | Simple, time-ordered data |
| **TTL** | Evicts after fixed duration | Data with known freshness window |

> Redis default is `noeviction`. Set `maxmemory-policy` to `allkeys-lru` or `volatile-lru` for production.

---

## Caching Layers

```
Browser Cache → CDN → Load Balancer → App Cache → DB Cache → Database
```

| Layer | What to Cache | Invalidation |
|-------|--------------|--------------|
| **Browser** | Static assets, API responses | Versioned URLs, Cache-Control |
| **CDN** | Static files, public API responses | Purge API, surrogate keys |
| **Application** | Computed results, DB queries, external API | Event-driven, TTL |
| **Database** | Query plans, buffer pool, materialized views | `ANALYZE`, manual refresh |

---

## Cache Stampede Prevention

When a hot key expires, hundreds of requests simultaneously hit the database.

| Technique | How It Works |
|-----------|-------------|
| **Mutex / Lock** | First request locks, fetches, populates; others wait |
| **Probabilistic early expiration** | Random chance of refreshing before TTL |
| **Request coalescing** | Deduplicate in-flight requests for same key |
| **Stale-while-revalidate** | Serve stale, refresh asynchronously |

---

## Cache Warming

| Strategy | When to Use |
|----------|-------------|
| **On-deploy warm-up** | Predictable key set, latency-sensitive |
| **Background job** | Reports, dashboards, catalog data |
| **Shadow traffic** | Cache migration, new infrastructure |
| **Priority-based** | Limited warm-up time budget |

> **Cold start impact:** A full cache flush can increase DB load 10–100x. Always warm gradually or use stale-while-revalidate.

---

## Monitoring

| Metric | Healthy Range | Action if Unhealthy |
|--------|--------------|---------------------|
| **Hit rate** | > 90% | Low → cache too small, wrong TTL, bad key design |
| **Eviction rate** | Near 0 steady state | High → increase memory or tune policy |
| **Latency (p99)** | < 1ms (Redis) | High → network issue, large values, hot key |
| **Memory usage** | < 80% of max | Approaching max → scale up or tune eviction |

---

## NEVER Do

1. **NEVER cache without a TTL or invalidation plan** — data rots; every entry needs an expiry path
2. **NEVER treat cache as durable storage** — caches evict, crash, and restart; always fall back to source of truth
3. **NEVER cache sensitive data (tokens, PII) without encryption** — cache breaches expose everything in plaintext
4. **NEVER ignore cache stampede on hot keys** — one expired popular key can take down your database
5. **NEVER use unbounded in-memory caches in production** — memory grows until OOM-killed
6. **NEVER cache mutable data with `immutable` Cache-Control** — browsers will never re-fetch
7. **NEVER skip monitoring hit/miss rates** — you won't know if your cache is helping or hurting