7.2 — Caching architecture

Module 7 · Lesson 2 · 🔴 Advanced · ~45 min

What you'll learn

• A complete cache layer map for a commerce site, with TTLs and invalidation per layer
• Stale‑while‑revalidate as a design principle, not just a header
• Surrogate keys and event‑driven invalidation that scales to 2.4M SKUs
• The failure modes: stampedes, leaks, staleness incidents — and how to survive them

---

The layer map

┌─────────────────────────────────────────────────────────────────────────┐
│ L0  Browser cache        immutable assets · 1y · invalidate by filename │
│ L1  Service worker       optional · app shell · [8.3]                   │
│ L2  CDN edge             HTML + API + images · SWR · surrogate keys     │
│ L3  Next.js Route Cache  rendered HTML/RSC · revalidate · revalidatePath│
│ L4  Next.js Data Cache   fetch results · revalidate · revalidateTag     │
│ L5  Application cache    Redis · shared across pods · explicit keys     │
│ L6  Database             query cache, materialized views, read replicas │
└─────────────────────────────────────────────────────────────────────────┘

Design principle: cache as close to the user as the correctness requirements allow. An L2 hit is 20 ms; an L6 hit is 200 ms plus render time. Every layer you push the hit outward is a 10× improvement.

Correctness principle: the closer to the user you cache, the harder invalidation is. You can purge Redis instantly; purging a browser cache is impossible. Match TTL to how wrong you can afford to be.

---

Per‑data caching decisions

Data	Layer	TTL	Invalidation	Staleness tolerance
Hashed JS/CSS	L0	1 year	Filename hash	∞ (content‑addressed)
Product images (versioned path)	L0+L2	1 year	URL version	∞
Product content	L2+L4	24 h	product-{id} webhook	Hours
Category tree	L4+L5	1 h	categories on publish	Minutes
Base price	L4	5 min	product-{id} on price event	Minutes
Promo/segment price	none	—	—	Zero — dynamic hole
Inventory	none, or L5 30 s	30 s	Time	Seconds (oversell risk)
Search results (top 2K queries)	L2	5 min	Time	Minutes
Search results (long tail)	none	—	—	—
Reviews summary	L4	1 h	product-{id} on new review	Hours
CMS content	L4	10 min	cms-{id} on publish	Minutes
Recommendations (per product)	L4+L5	30 min	Time	Minutes
Cart	none	—	—	Zero
User/session	none	—	—	Zero
Order history	none	—	—	Zero

The rule that prevents the career‑ending bug:

Never cache anything derived from a session, a user ID, or a cookie — at any layer. The failure is one customer seeing another's cart or address. It's a reportable breach.

---

Stale‑while‑revalidate as a design principle

The most valuable idea in this lesson. Instead of "fresh or slow", you get "fast always, fresh soon".

Cache-Control: public, s-maxage=300, stale-while-revalidate=86400

t=0      Response cached
t=0-300  Fresh: served from cache instantly                  20ms
t=300+   Stale: served from cache instantly (20ms) AND
         refreshed in the background for the next request
t=86400+ Expired: must fetch from origin                     600ms

With s-maxage=300, stale-while-revalidate=86400, effectively every user gets a 20 ms response, and content is at most ~5 minutes stale (plus one background refresh cycle).

Apply it everywhere staleness is tolerable:

// Route Handler
return Response.json(data, {
  headers: {
    'Cache-Control': 'public, s-maxage=300, stale-while-revalidate=86400',
    'Surrogate-Key': `product-${id} category-${categoryId}`,
  },
});

// SWR on the client for the same principle
useSWR(key, fetcher, {
  fallbackData: serverRendered,     // instant, from the server
  revalidateOnMount: false,          // don't refetch what the server just gave us
  refreshInterval: 60_000,
});

When SWR is wrong: anything where showing stale data causes harm. Inventory ("in stock" when it isn't), prices during a flash sale, order status, and anything legally binding.

---

Surrogate keys

Purging by URL doesn't work when one product appears on 40 category pages, the homepage, and 6 search result pages. Tag responses with everything they contain, then purge by tag.

// lib/cache-tags.ts
export function productTag(id: string) { return `product-${id}`; }
export function categoryTag(id: string) { return `category-${id}`; }
export function brandTag(id: string) { return `brand-${id}`; }
export function cmsTag(id: string) { return `cms-${id}`; }
export function marketTag(code: string) { return `market-${code}`; }

// app/c/[slug]/page.tsx — tag a PLP with every product it renders
import { headers } from 'next/headers';

// In a Route Handler or via a response header helper:
const keys = [
  categoryTag(category.id),
  ...products.map((p) => productTag(p.id)),
  marketTag(market),
].join(' ');
// Surrogate-Key: category-123 product-1 product-2 … market-uk

// lib/purge.ts — one purge call reaches every page containing that product
export async function purgeSurrogateKeys(keys: string[]) {
  const unique = [...new Set(keys)];
  // Chunk: most CDN APIs cap keys per request
  for (let i = 0; i < unique.length; i += 100) {
    await fetch(`https://api.cdn-vendor.com/service/${SERVICE_ID}/purge`, {
      method: 'POST',
      headers: {
        'Fastly-Key': process.env.CDN_API_KEY!,
        'Content-Type': 'application/json',
        'fastly-soft-purge': '1',        // mark stale, don't evict
      },
      body: JSON.stringify({ surrogate_keys: unique.slice(i, i + 100) }),
    });
  }
}

Soft purge is the default you want. It marks entries stale rather than deleting them, so users keep getting instant (slightly stale) responses while the CDN refreshes. Hard purge sends every request for that content to your origin simultaneously — a self‑inflicted stampede at exactly the moment (a price change, a campaign launch) when traffic is highest.

---

Event‑driven invalidation

Catalog/PIM/Pricing systems
        │ events
        ▼
   Event bus (Kafka/SNS)
        │
        ▼
  Invalidation worker
   · dedupe within a window
   · batch
   · rate limit
   · fan out to all layers
        │
        ├──▶ revalidateTag()      (L4 Data Cache)
        ├──▶ revalidatePath()     (L3 Route Cache)
        ├──▶ purgeSurrogateKeys() (L2 CDN)
        └──▶ redis.del()          (L5)

// workers/invalidation.ts
const WINDOW_MS = 5_000;
const MAX_PER_FLUSH = 200;

const pending = new Map<string, InvalidationEvent>();
let timer: NodeJS.Timeout | null = null;

export function onCatalogEvent(event: InvalidationEvent) {
  // Dedupe: a bulk update sends the same product many times
  pending.set(keyFor(event), event);
  timer ??= setTimeout(flush, WINDOW_MS);
}

async function flush() {
  timer = null;
  const batch = [...pending.values()].slice(0, MAX_PER_FLUSH);
  batch.forEach((e) => pending.delete(keyFor(e)));

  const tags = batch.flatMap(tagsFor);

  // All layers, in the right order: innermost first, so a refill after the
  // outer purge reads fresh data.
  await redis.del(...batch.map(redisKeyFor));
  await postSigned('/api/revalidate', { tags });        // L3 + L4
  await purgeSurrogateKeys(tags);                        // L2

  if (pending.size) timer = setTimeout(flush, 1_000);    // keep draining
}

Order matters. Purge inner layers before outer ones. If you purge the CDN first, a request can arrive, miss at the CDN, hit the still‑stale Data Cache, and re‑cache the stale value at the edge — and now you've "invalidated" nothing.

---

Cache stampedes

Popular PDP entry expires
  → 4,000 concurrent requests miss
  → 4,000 origin renders for the same page
  → origin saturates
  → latency spikes for everything
  → more timeouts, more retries
  → outage

Four defenses:

1. stale-while-revalidate — expiry no longer means a miss.
2. Origin shielding / request collapsing at the CDN — one origin request per key. Most CDNs support this; verify yours is enabled.
3. Distributed single‑flight at the origin (3.5).
4. Jittered TTLs so 20,000 entries don't expire in the same second.

export function jitter(seconds: number, pct = 0.2) {
  const spread = seconds * pct;
  return Math.round(seconds - spread / 2 + Math.random() * spread);
}

Test it. Nobody discovers a stampede problem gradually; you discover it during a flash sale.

# Expire a hot key, then hit it hard. Count origin renders.
redis-cli DEL "next:/p/wool-overshirt-navy"
npx autocannon -c 500 -d 10 https://staging.auroramarket.com/p/wool-overshirt-navy
# Then check your origin's request count for that path: should be ~1, not ~500.

---

Redis as the shared application cache (L5)

For self‑hosted multi‑instance deployments, L5 is what makes the other layers work.

// lib/cache.ts
import 'server-only';
import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

type CacheOptions = { ttlSeconds: number; tags?: string[]; jitterPct?: number };

export async function cached<T>(
  key: string,
  fn: () => Promise<T>,
  { ttlSeconds, tags = [], jitterPct = 0.2 }: CacheOptions,
): Promise<T> {
  const hit = await redis.get(key);
  if (hit) return JSON.parse(hit) as T;

  // Single-flight: only one instance computes on a miss
  const lockKey = `lock:${key}`;
  const gotLock = await redis.set(lockKey, '1', { NX: true, PX: 10_000 });

  if (!gotLock) {
    await new Promise((r) => setTimeout(r, 100));
    const retry = await redis.get(key);
    if (retry) return JSON.parse(retry) as T;
    // Still nothing — compute anyway rather than stalling the request
  }

  try {
    const value = await fn();
    const ttl = jitter(ttlSeconds, jitterPct);
    await redis.set(key, JSON.stringify(value), { EX: ttl });
    // Tag index for fan-out invalidation
    for (const tag of tags) {
      await redis.sAdd(`tag:${tag}`, key);
      await redis.expire(`tag:${tag}`, ttl * 2);
    }
    return value;
  } finally {
    if (gotLock) await redis.del(lockKey);
  }
}

export async function invalidateTags(tags: string[]) {
  for (const tag of tags) {
    const keys = await redis.sMembers(`tag:${tag}`);
    if (keys.length) await redis.del(keys);
    await redis.del(`tag:${tag}`);
  }
}

Operational rules for L5:

• Never let a cache failure take down the site. Wrap reads in try/catch and fall through to the origin. A Redis outage should be a latency event, not an availability event.
• Set maxmemory-policy allkeys-lru so you degrade rather than OOM.
• Monitor hit ratio per key prefix. A prefix under 70% is either wrongly keyed or wrongly TTL'd.
• Watch key cardinality. A key that includes a user ID or a full query string will fill Redis and evict everything useful.

// Fail open, always
export async function cachedSafe<T>(key: string, fn: () => Promise<T>, opts: CacheOptions) {
  try {
    return await cached(key, fn, opts);
  } catch (err) {
    logWarn('cache_unavailable', { key, err });
    return fn();          // degrade to origin
  }
}

---

Monitoring the cache

Five metrics, on one dashboard:

Metric	Target	Alert
CDN HTML hit ratio (per path pattern)	> 90% on /p/**, /c/**	< 70%
CDN asset hit ratio	> 98%	< 95%
Data Cache / Redis hit ratio	> 85%	< 70%
Origin requests per second	Stable	2× baseline
Invalidation events per minute	Stable	10× baseline (runaway purge)

The last one catches a specific incident: a bug in the invalidation worker that purges everything in a loop. Symptoms are a hit‑ratio collapse and an origin overload, and it's easy to misdiagnose as a traffic spike.

// Emit cache status from your app so it's queryable in RUM
response.headers.set('x-cache-status', hit ? 'HIT' : 'MISS');
response.headers.set('x-cache-age', String(ageSeconds));

---

Staleness incidents

You will eventually show a wrong price. Plan for it.

Prevention: - Promotional/segment prices are never cached — dynamic hole only (3.6) - Base price TTL ≤ 5 minutes with event‑driven purge - Cart re‑prices server‑side on every view. The PDP price is marketing; the cart price is the contract. This is the control that turns a stale price from an incident into a mild annoyance. - Checkout re‑validates prices and inventory before payment authorization

Detection:

// A synthetic check comparing the rendered price against the source of truth
// Run every 5 minutes against a sample of SKUs.
for (const sku of SAMPLE_SKUS) {
  const [rendered, canonical] = await Promise.all([
    scrapeRenderedPrice(`/p/${sku.slug}`),
    pricingService.get(sku.id),
  ]);
  if (Math.abs(rendered - canonical.amount) > 0.01) {
    alert('price_mismatch', { sku: sku.id, rendered, canonical: canonical.amount });
  }
}

Response runbook: 1. Confirm which layer is stale (check age and x-cache at each layer). 2. Purge that layer's tag for the affected SKUs. 3. If widespread, purge the category or market tag. 4. Only as a last resort, purge everything — it will spike origin load, so scale up first. 5. Post‑incident: why didn't the event fire? Was it dropped, batched away, or never emitted?

---

Common mistakes

Mistake	Cost
Caching anything session‑derived	🚨 Cross‑user data leak
Hard purge instead of soft	Origin stampede at the worst moment
Purging outer layers before inner	Stale data immediately re‑cached
No jitter	Synchronized expiry storms
Cache failure taking down the site	Availability incident from a latency component
High‑cardinality cache keys	Redis fills; useful entries evicted
No hit‑ratio monitoring per prefix	Silent cache degradation for months
Trusting the PDP price at checkout	Wrong prices become real orders
No invalidation‑rate alert	Runaway purge looks like a traffic spike

---

Lab 7.2 — Cache architecture review

1. Draw your layer map. For every data type, which layers cache it, with what TTL and what invalidation? Most teams discover an undocumented layer doing something surprising.
2. Verify the no‑personalization rule with the cross‑user test from 3.6. Put it in CI.
3. Measure hit ratios per layer and per path pattern. Anything below target, find out why.
4. Test a stampede in staging with autocannon. Count origin renders.
5. Test invalidation end to end: change a price in the source system, then measure how long until every layer reflects it. Aurora's target is 60 seconds; measure yours.
6. Add the price‑mismatch synthetic check and the invalidation‑rate alert.
7. Fail‑open test: kill Redis in staging and confirm the site degrades to slow, not down.

---

Checklist

• Layer map documented with TTL and invalidation per data type
• Nothing session‑derived cached at any layer, enforced by a CI test
• SWR on everything with staleness tolerance
• Surrogate keys on all cacheable responses
• Soft purge as the default
• Invalidation batched, deduped, rate‑limited, inner‑layer‑first
• Jittered TTLs
• Stampede protection tested under load
• Cache failures degrade to slow, never to down
• Hit ratios monitored per layer and prefix, with alerts
• Cart and checkout re‑price server‑side
• Price‑mismatch synthetic check running

Next: 7.3 Search & catalog data