9.2 — CI gates & synthetic monitoring

Module 9 · Lesson 2 · 🟡 Intermediate · ~40 min

What you'll learn

• What to block a PR on, and what to merely warn about
• A complete GitHub Actions setup: bundle budgets, Lighthouse CI, interaction tests
• Synthetic monitoring for production, and why it complements RUM
• How to keep gates from becoming the thing everyone routes around

---

The gate hierarchy

Not everything deserves to block a merge. Match the enforcement to the signal quality.

Signal	Determinism	Gate
Bundle size per route	Perfect	❌ Block
Route rendering mode (static vs dynamic)	Perfect	❌ Block
RSC payload size	Perfect	❌ Block
'use client' file count (ratchet)	Perfect	❌ Block
New third‑party origins	Perfect	❌ Block
Expired feature flags	Perfect	❌ Block
Lighthouse LCP/TBT/CLS	±5–10% noise	⚠️ Warn, block on large regressions
Playwright interaction latency	±15% noise	⚠️ Warn, block on 2× regressions
Field p75	Slow, noisy	📊 Dashboard + alert, never a PR gate

The principle: block on deterministic proxies, warn on noisy outcomes, alert on field data. A gate that fails randomly gets disabled within a month, taking the useful gates with it.

---

The workflow

# .github/workflows/performance.yml
name: Performance

on:
  pull_request:
    branches: [main]

concurrency:
  group: perf-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build-and-measure:
    runs-on: ubuntu-latest
    timeout-minutes: 25
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }

      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }

      - run: npm ci

      # ── Deterministic gates: these BLOCK ────────────────────────────────
      - name: Build
        run: npm run build
        env: { NEXT_TELEMETRY_DISABLED: 1 }

      - name: Bundle budgets
        run: node scripts/bundle-budget.mjs

      - name: Route rendering modes
        run: node scripts/check-route-modes.mjs

      - name: Client boundary ratchet
        run: node scripts/check-client-boundaries.mjs

      - name: Third-party origins
        run: node scripts/check-third-party-origins.mjs

      - name: Expired feature flags
        run: node scripts/check-expired-flags.mjs

      # ── Noisy gates: these WARN ─────────────────────────────────────────
      - name: Start server
        run: |
          npm start &
          npx wait-on http://localhost:3000 -t 60000

      - name: Lighthouse CI
        run: npx @lhci/cli autorun
        continue-on-error: true          # warn, don't block
        env: { LHCI_GITHUB_APP_TOKEN: '${{ secrets.LHCI_GITHUB_APP_TOKEN }}' }

      - name: Interaction latency
        run: npx playwright test tests/performance/
        continue-on-error: true

      # ── Report ──────────────────────────────────────────────────────────
      - name: Comment on PR
        if: always()
        run: node scripts/post-perf-comment.mjs
        env: { GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}' }

---

Gate 1 — Bundle budgets

The highest‑value gate. See examples/ci/bundle-budget.mjs for the full implementation.

// scripts/bundle-budget.mjs (abridged)
import { readFileSync, statSync } from 'node:fs';
import { gzipSync } from 'node:zlib';
import { join } from 'node:path';

const BUDGETS_KB = {
  '/':                200,
  '/c/[slug]':        240,
  '/p/[slug]':        260,
  '/search':          240,
  '/cart':            200,
  '/checkout/[step]': 180,
  __shared__:         150,
  __default__:        240,
};

const manifest = JSON.parse(readFileSync('.next/app-build-manifest.json', 'utf8'));

function gzipKb(file) {
  return gzipSync(readFileSync(join('.next', file)), { level: 9 }).length / 1024;
}

const results = [];
for (const [route, files] of Object.entries(manifest.pages)) {
  const kb = files.filter((f) => f.endsWith('.js')).reduce((s, f) => s + gzipKb(f), 0);
  const budget = BUDGETS_KB[route] ?? BUDGETS_KB.__default__;
  results.push({ route, kb: +kb.toFixed(1), budget, over: kb > budget });
}

console.table(results);

const failures = results.filter((r) => r.over);
if (failures.length) {
  console.error('\n❌ Bundle budget exceeded:');
  for (const f of failures) {
    console.error(`   ${f.route}: ${f.kb} KB > ${f.budget} KB (+${(f.kb - f.budget).toFixed(1)})`);
  }
  console.error('\nOptions: reduce the code, lazy-load it, or file a dated budget exception.');
  console.error('See docs/01-foundations/05-performance-budgets.md');
  process.exit(1);
}
console.log('\n✅ All routes within budget');

Gzip the files yourself rather than trusting reported sizes — it's the number your users actually download, and it's what your budget is denominated in.

---

Gate 2 — Route rendering modes

Catches the highest‑impact silent regression: a route becoming dynamic.

// scripts/check-route-modes.mjs
import { readFileSync } from 'node:fs';

const EXPECTED = {
  '/':                 'static',
  '/c/[slug]':         'static',    // ISR
  '/p/[slug]':         'partial',   // PPR
  '/search':           'dynamic',
  '/cart':             'dynamic',
  '/checkout/[step]':  'dynamic',
};

const prerender = JSON.parse(readFileSync('.next/prerender-manifest.json', 'utf8'));
const appRoutes = JSON.parse(readFileSync('.next/app-path-routes-manifest.json', 'utf8'));

function classify(route) {
  if (prerender.dynamicRoutes?.[route]) {
    return prerender.dynamicRoutes[route].experimentalPPR ? 'partial' : 'static';
  }
  if (Object.values(prerender.routes ?? {}).some((r) => r.srcRoute === route)) return 'static';
  return 'dynamic';
}

const failures = [];
for (const [route, expected] of Object.entries(EXPECTED)) {
  const actual = classify(route);
  if (actual !== expected) failures.push({ route, expected, actual });
}

if (failures.length) {
  console.error('❌ Route rendering mode changed:\n');
  for (const f of failures) {
    console.error(`   ${f.route}: expected "${f.expected}", got "${f.actual}"`);
  }
  console.error('\nA route becoming dynamic costs 400-900ms of TTFB.');
  console.error('Look for a new cookies() / headers() / searchParams / no-store fetch.');
  console.error('See docs/03-rendering/01-choosing-a-rendering-strategy.md');
  process.exit(1);
}

Manifest shapes vary across Next.js versions. Verify the classification logic against your version once, then it's stable.

---

Gate 3 — Client boundary ratchet

// scripts/check-client-boundaries.mjs
import { execSync } from 'node:child_process';
import { readFileSync, writeFileSync } from 'node:fs';

const BASELINE_FILE = '.perf-baseline.json';

const count = Number(
  execSync(`grep -rl "^'use client'" app components | wc -l`, { encoding: 'utf8' }).trim(),
);

const baseline = JSON.parse(readFileSync(BASELINE_FILE, 'utf8'));

if (count > baseline.clientComponents) {
  console.error(
    `❌ Client Component count increased: ${baseline.clientComponents} → ${count}\n\n` +
    `Every 'use client' file adds bundle weight and hydration cost.\n` +
    `Can the interactive part be a smaller leaf? See docs/03-rendering/02-server-components-and-boundaries.md\n\n` +
    `If this is intentional, update ${BASELINE_FILE} in this PR with a note in the description.`,
  );
  process.exit(1);
}

if (count < baseline.clientComponents) {
  // Ratchet down automatically — improvements lock in
  writeFileSync(BASELINE_FILE, JSON.stringify({ ...baseline, clientComponents: count }, null, 2));
  console.log(`✅ Client Component count reduced: ${baseline.clientComponents} → ${count}`);
}

A ratchet is the right shape for any "should trend down" metric: it never blocks improvement, it requires a deliberate decision to regress, and it locks in gains automatically.

---

Gate 4 — Lighthouse CI (warn)

// lighthouserc.js
module.exports = {
  ci: {
    collect: {
      url: [
        'http://localhost:3000/',
        'http://localhost:3000/c/womens-knitwear',
        'http://localhost:3000/p/wool-overshirt-navy',
        'http://localhost:3000/cart',
      ],
      numberOfRuns: 5,                     // median of 5 — never a single run
      settings: {
        preset: 'desktop',
        skipAudits: ['uses-http2', 'canonical', 'is-crawlable'],  // noisy on localhost
        throttlingMethod: 'simulate',
      },
    },
    assert: {
      preset: 'lighthouse:no-pwa',
      assertions: {
        // Outcome metrics: generous thresholds, because of run variance
        'largest-contentful-paint': ['warn', { maxNumericValue: 2500 }],
        'total-blocking-time':      ['warn', { maxNumericValue: 300 }],
        'cumulative-layout-shift':  ['error', { maxNumericValue: 0.05 }],  // CLS is stable enough to block

        // Deterministic diagnostics: these CAN block
        'uses-responsive-images':   ['error', { minScore: 0.9 }],
        'modern-image-formats':     ['error', { minScore: 0.9 }],
        'unsized-images':           ['error', { minScore: 1 }],
        'prioritize-lcp-image':     ['error', { minScore: 0.9 }],
        'render-blocking-resources':['warn',  { maxNumericValue: 300 }],
        'unused-javascript':        ['warn',  { maxNumericValue: 120000 }],
        'third-party-summary':      ['warn',  { maxNumericValue: 500 }],
      },
    },
    upload: { target: 'temporary-public-storage' },
  },
};

Note which assertions block. CLS and the image diagnostics are deterministic enough to gate; LCP and TBT are not, in a CI environment on a shared runner.

---

Gate 5 — Interaction latency (warn)

Lighthouse can't measure INP. These tests approximate it deterministically.

// tests/performance/interactions.spec.ts
import { test, expect, type Page } from '@playwright/test';

async function throttle(page: Page, rate = 4) {
  const client = await page.context().newCDPSession(page);
  await client.send('Emulation.setCPUThrottlingRate', { rate });
}

/** Measures interaction → next painted frame, the way INP does. */
async function timeInteraction(page: Page, selector: string) {
  return page.evaluate(async (sel) => {
    const el = document.querySelector<HTMLElement>(sel);
    if (!el) throw new Error(`Not found: ${sel}`);
    const t0 = performance.now();
    el.click();
    await new Promise<void>((r) => requestAnimationFrame(() => requestAnimationFrame(() => r())));
    return performance.now() - t0;
  }, selector);
}

const CASES = [
  { name: 'filter toggle',   path: '/c/womens-knitwear',   selector: '[data-testid="facet-color-navy"]', budget: 200 },
  { name: 'variant select',  path: '/p/wool-overshirt-navy', selector: '[data-testid="swatch-1"]',       budget: 200 },
  { name: 'add to cart',     path: '/p/wool-overshirt-navy', selector: '[data-testid="add-to-cart"]',    budget: 250 },
  { name: 'open mini-cart',  path: '/c/womens-knitwear',   selector: '[data-testid="cart-toggle"]',      budget: 200 },
];

for (const c of CASES) {
  test(`${c.name} under ${c.budget}ms`, async ({ page }) => {
    await throttle(page);
    await page.goto(c.path);
    await page.waitForLoadState('networkidle');

    // Median of 3 to reduce noise
    const samples: number[] = [];
    for (let i = 0; i < 3; i++) {
      samples.push(await timeInteraction(page, c.selector));
      await page.waitForTimeout(300);
    }
    samples.sort((a, b) => a - b);
    const median = samples[1];

    console.log(`${c.name}: ${median.toFixed(0)}ms (samples: ${samples.map(s => s.toFixed(0))})`);
    expect(median).toBeLessThan(c.budget);
  });
}

Add the CLS test from 6.3 and the memory test from 8.4 to the same suite.

---

The PR comment

A red X with no explanation makes enemies. The comment is what makes the gate useful.

// scripts/post-perf-comment.mjs (abridged)
const body = `
## 📊 Performance report

### Bundle size
| Route | Base | PR | Δ | Budget | |
|---|---|---|---|---|---|
${rows.map(r =>
  `| \`${r.route}\` | ${r.base} KB | ${r.pr} KB | ${r.delta > 0 ? '+' : ''}${r.delta} KB | ${r.budget} KB | ${r.status} |`
).join('\n')}

${newModules.length ? `### New modules in this PR
${newModules.map(m => `- \`${m.name}\` **${m.kb} KB** — imported in ${m.importedBy}`).join('\n')}
` : ''}

### Lighthouse (median of 5, desktop)
| Metric | Base | PR | Δ |
|---|---|---|---|
| LCP | ${lh.base.lcp} ms | ${lh.pr.lcp} ms | ${lh.delta.lcp} |
| TBT | ${lh.base.tbt} ms | ${lh.pr.tbt} ms | ${lh.delta.tbt} |
| CLS | ${lh.base.cls} | ${lh.pr.cls} | ${lh.delta.cls} |

### Interaction latency (4× CPU, median of 3)
${interactions.map(i => `- ${i.name}: **${i.ms} ms** (budget ${i.budget} ms) ${i.pass ? '✅' : '❌'}`).join('\n')}

${suggestions.length ? `### 💡 Suggestions
${suggestions.map(s => `- ${s}`).join('\n')}` : ''}

<sub>[Performance course](docs/README.md) · [Budgets](docs/01-foundations/05-performance-budgets.md) · [Request an exception](docs/01-foundations/05-performance-budgets.md#when-a-budget-should-break)</sub>
`;

The suggestions section is what makes people fix things instead of asking for exceptions:

💡 Suggestions
- `react-image-gallery` (18.2 KB) is below the fold on /p/[slug] — consider next/dynamic
- `date-fns/locale` barrel import in ReviewDate.tsx:2 — import the single locale you need
- 3 new images without explicit dimensions — see docs/06-web-vitals-playbooks/03-cls.md

---

Synthetic monitoring in production

CI tests pre‑merge code. Synthetics test what's actually live, continuously.

# .github/workflows/synthetic.yml
name: Synthetic monitoring
on:
  schedule: [{ cron: '0 * * * *' }]      # hourly
  workflow_dispatch:

jobs:
  monitor:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        page: ['/', '/c/womens-knitwear', '/p/wool-overshirt-navy', '/cart']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: |
          npx lighthouse "https://www.auroramarket.com${{ matrix.page }}" \
            --form-factor=mobile --output=json --output-path=./result.json \
            --chrome-flags="--headless=new"
      - run: node scripts/push-synthetic-metrics.mjs ./result.json "${{ matrix.page }}"

Why synthetics as well as RUM:

	RUM	Synthetics
Reflects real users	✅	❌
Consistent baseline	❌ (traffic mix shifts)	✅
Detects regressions fast	Hours	Minutes
Works with zero traffic	❌	✅ (staging, new pages)
Isolates a variable	❌	✅

The critical synthetic checks for commerce:

1. Vitals per page type, hourly
2. Critical user journey — search → PDP → add to cart → checkout — every 15 minutes, with step timings
3. Cache health — is x-cache: HIT on cacheable pages?
4. Route mode — is the PDP still returning a cached shell?
5. Price correctness — rendered price vs the source of truth (7.2)

// tests/synthetic/journey.spec.ts
test('critical purchase journey', async ({ page }) => {
  const timings: Record<string, number> = {};
  const mark = (name: string, t0: number) => { timings[name] = Date.now() - t0; };

  let t = Date.now();
  await page.goto('https://www.auroramarket.com/');
  mark('home', t);

  t = Date.now();
  await page.fill('[data-testid="search"]', 'wool overshirt');
  await page.press('[data-testid="search"]', 'Enter');
  await page.waitForSelector('[data-testid="product-tile"]');
  mark('search', t);

  t = Date.now();
  await page.click('[data-testid="product-tile"]:first-child');
  await page.waitForSelector('[data-testid="add-to-cart"]');
  mark('pdp', t);

  t = Date.now();
  await page.click('[data-testid="add-to-cart"]');
  await page.waitForSelector('[data-testid="cart-count"]:not(:empty)');
  mark('add_to_cart', t);

  await reportSyntheticJourney(timings);

  expect(timings.pdp).toBeLessThan(3000);
  expect(timings.add_to_cart).toBeLessThan(1500);
});

This catches the class of failure that vitals miss entirely: the site is fast but add‑to‑cart is broken. That's a far worse outage than a slow LCP, and it's often invisible in performance dashboards.

---

Keeping gates alive

Gates die when they're noisy or unhelpful. Five practices:

1. Fix flakiness immediately. A gate that fails randomly twice gets ignored forever.
2. Always explain. Every failure message says what's wrong, why it matters, and what to do.
3. Provide an escape hatch — a dated exception process (1.5).
4. Keep them fast. Over ~10 minutes and people start merging without waiting.
5. Review quarterly. Which gate has caught real problems? Which only produces noise? Delete the latter.

Quarterly gate review — Q1
Gate                     Fired   True positives   Action
bundle-budget              23         19          Keep
route-modes                 4          4          Keep — caught 2 TTFB regressions
client-boundary-ratchet    11          8          Keep
lighthouse-lcp             41          3          Demote to warn (already warn) — consider removing
lighthouse-cls              7          7          Keep, promote to blocking ✅
interaction-latency        14          9          Keep
third-party-origins         2          2          Keep

---

Common mistakes

Mistake	Cost
Blocking on noisy Lighthouse metrics	Random failures; gates get disabled
Single Lighthouse run	±5–10 points of noise
No base‑branch comparison	"It was already big" instead of "you added 27 KB"
Failure with no explanation	Resentment, exception requests instead of fixes
Gates taking 20 minutes	People merge without waiting
No exception process	Team routes around the gate
Gates never reviewed	Noise accumulates; signal is lost
Synthetics without a journey test	You'll miss "fast but broken"

---

Lab 9.2 — Build the pipeline

1. Ship the bundle budget gate first. It's the highest value and the easiest to get right.
2. Add the route‑mode gate. It catches the most expensive silent regression.
3. Add the client boundary ratchet with a committed baseline file.
4. Set up Lighthouse CI with median‑of‑5 and the assertion split above (warn on outcomes, block on diagnostics).
5. Write three interaction latency tests for your top interactions.
6. Build the PR comment, including the suggestions section.
7. Set up hourly synthetics plus a 15‑minute journey test.
8. Schedule the quarterly gate review in your team calendar now.

---

Checklist

• Deterministic gates block; noisy ones warn
• Bundle budgets per route, gzipped, compared against the base branch
• Route rendering mode gated
• Client Component count ratcheted
• Third‑party origin allowlist enforced
• Lighthouse median‑of‑5, blocking only on deterministic audits
• Interaction latency tests at 4× CPU
• PR comment explains failures and suggests fixes
• Synthetics on vitals, cache health, and a full purchase journey
• Exception process documented and used
• Total CI time under ~10 minutes
• Quarterly gate review scheduled

Next: 9.3 Regression triage