Running in Production

Chapter 5 | Getting Claude Code to Finish Work at Production Grade Written for: developers at every level who want to ship code to production with confidence

From vibe coding to production — the gap you have to cross

Chapter 4 taught you to build things fast, but “fast” is not always “ready”. Production code has to go through a complete process before real users touch it. This chapter walks you through everything from freshly vibed code to a system that is deployed for real and can be measured.

The production gap — what vibe coding still lacks

What vibe coding gives you	What production adds
The function works	Every possible edge case is handled
The happy path passes	An error path that is graceful and informative
The code is readable	Code another person can maintain 6 months later
It runs locally	It runs consistently in every environment
Tested by hand	Automated tests that run on every commit
The first deploy works	Repeat deploys that are reliable, however many times
No idea whether it is fast	A clear benchmark and performance budget
No idea whether users like it	An A/B test that measures with real data

The full production pipeline

This is the pipeline every feature passes through before reaching a real user. Claude Code helps at every stage:

Stage	Phase	Main work	How Claude helps
1	Code quality	Lint, format, type check	Auto-fix + review
2	Unit testing	Test every function, every edge case	Writes + runs the tests
3	Integration testing	Test that components work together	Builds test scenarios
4	Security audit	Find holes before deploying	OWASP checklist
5	Performance	Benchmark + optimise	Finds the bottleneck
6	QA testing	Manual + automated E2E	Builds the test plan
7	Staging deploy	Test in a real environment	Deployment script
8	Monitoring setup	Logs, metrics, alerts	Writes the dashboard
9	Production deploy	Release strategy	Blue/green, canary
10	A/B testing	Measure the new feature	Builds the experiment

Phase 1: Code quality — the standard that comes before everything

Code that fails the quality gate should not move on. Configure it to block automatically, so bad code never enters the repository.

Setting up an automatic quality gate

Have Claude set up linting, formatting and type checking with zero configuration on your part:

> Set up a code quality toolchain for Next.js + TypeScript:
  1. ESLint: use the next/typescript config + strict rules
  2. Prettier: single quote, no semi, 100 char line length
  3. TypeScript: strict mode, no implicit any
  4. Husky pre-commit hook: run lint + typecheck before a commit
  5. lint-staged: run prettier only on changed files (faster)
 
  Install and configure everything in one pass
  Give a clear error when a commit does not pass

# the package.json scripts you should have
{
  "scripts": {
    "lint": "next lint --fix",
    "lint:check": "next lint",
    "typecheck": "tsc --noEmit",
    "format": "prettier --write .",
    "format:check": "prettier --check .",
    "quality": "npm run typecheck && npm run lint:check && npm run format:check"
  },
  "lint-staged": {
    "*.{ts,tsx}": ["eslint --fix", "prettier --write"],
    "*.{json,md,css}": ["prettier --write"]
  }
}
 
# .husky/pre-commit
#!/bin/sh
npx lint-staged
npm run typecheck

A Claude code review before the PR

Have Claude review the code every time before you open a pull request, using a checklist that covers the ground:

# .claude/commands/pr-review.md
# Pre-PR Code Review
 
Review every change made since main:
 
## 1. Logic correctness
- Are the business rules correct per @docs/domain/business-rules.md?
- Are the edge cases covered? (empty array, null, undefined, negative numbers)
- Boundary conditions? (0, max int, empty string)
 
## 2. Error Handling
- Does every async operation have a try/catch?
- Are the error messages meaningful?
- Do errors reach Sentry with enough context?
- Does the user see a friendly error rather than a stack trace?
 
## 3. Security
- Is input validation present on every endpoint?
- Is there an authorization check before every data access?
- Is sensitive data kept out of the logs?
- Are SQL/NoSQL injections prevented?
 
## 4. Performance
- N+1 queries?
- Missing database indexes?
- Unnecessary re-renders in React?
- Heavy computation in the render path?
 
## 5. TypeScript
- Any unnecessary use of any?
- Do the types cover null/undefined?
- Are generic types used correctly?
 
## Output Format
### ✅ APPROVED / ⚠️ APPROVED WITH NOTES / ❌ CHANGES REQUIRED
### Critical Issues (must be fixed before merge):
### Minor Issues (can be fixed later):
### Suggestions (optional):

Phase 2: Testing strategy — every layer covered

The testing pyramid is a sound principle: many unit tests, a moderate number of integration tests, and few E2E tests that nonetheless cover the critical paths. Having Claude write the tests raises coverage while taking less of your time.

The testing pyramid for a vibe coder

Layer	Count	Speed	Cost	Used for
Unit tests	Many (70%)	Fast, ms	Cheap	Every function, every utility
Integration tests	Middling (20%)	Medium, sec	Medium	API endpoints, DB queries
E2E tests	Few (10%)	Slow, minutes	Expensive	Critical user journeys only

Unit testing — have Claude write thorough tests

A good unit test covers the happy path, the error path, edge cases and boundary conditions:

> Write comprehensive unit tests for
  every function in @src/lib/pricing.ts
 
  Use Vitest + the describe/it pattern
  Cover:
  1. Happy path: normal input, correct output
  2. Edge cases: 0, negative, null, undefined, empty
  3. Boundary: the lowest/highest valid values
  4. Error cases: badly formatted input must throw
  5. Business rules: per @docs/domain/business-rules.md
 
  Format: describe("functionName") > it("should...")
  No magic numbers; explain each test case

An example of a good unit test

// pricing.test.ts
import { describe, it, expect } from "vitest"
import { calculateDiscount, applyMemberDiscount } from "./pricing"
 
describe("calculateDiscount", () => {
  describe("Coupon discount", () => {
    it("should apply percentage coupon correctly", () => {
      expect(calculateDiscount(1000, { type: "percent", value: 10 })).toBe(900)
    })
 
    it("should apply fixed amount coupon correctly", () => {
      expect(calculateDiscount(1000, { type: "fixed", value: 150 })).toBe(850)
    })
 
    it("should not go below minimum price", () => {
      // a 100% coupon, but there is a minimum price of 100
      const result = calculateDiscount(500, { type: "percent", value: 100 }, { minPrice: 100 })
      expect(result).toBe(100)
    })
 
    it("should throw when coupon value is negative", () => {
      expect(() => calculateDiscount(1000, { type: "percent", value: -10 }))
        .toThrow("Coupon value must be positive")
    })
 
    it("should throw when original price is zero", () => {
      expect(() => calculateDiscount(0, { type: "percent", value: 10 }))
        .toThrow("Original price must be greater than 0")
    })
  })
 
  describe("Member discount stacking", () => {
    it("should apply member discount AFTER coupon", () => {
      // 1000 → coupon 10% → 900 → member 5% → 855
      const afterCoupon = calculateDiscount(1000, { type: "percent", value: 10 })
      expect(applyMemberDiscount(afterCoupon, "gold")).toBe(855)
    })
  })
})

Integration testing — exercising the API endpoints

An integration test verifies that several components work together correctly, the real database included:

> Build integration tests for POST /api/orders
 
  Setup:
  - Use Vitest + supertest
  - Database: a test database separate from dev
  - Seed: create the user and products before the test
  - Cleanup: delete the data after every test
 
  Test cases:
  1. Order created successfully → 201 + order ID
  2. Order created with no auth → 401
  3. Product out of stock → 400 + error message
  4. Product does not exist → 404
  5. Payment amount does not match → 400
  6. Concurrent orders (race condition test)
 
  Verify with:
  - The response status code
  - The response body structure
  - The database state after the request
  - Side effects (email sent? inventory updated?)

// orders.integration.test.ts
import { describe, it, expect, beforeEach, afterEach } from "vitest"
import request from "supertest"
import { app } from "../app"
import { db } from "../lib/db"
import { createTestUser, createTestProduct, clearTestData } from "./helpers"
 
describe("POST /api/orders", () => {
  let authToken: string
  let testProduct: { id: string; price: number; stock: number }
 
  beforeEach(async () => {
    const user = await createTestUser({ email: "test@test.com" })
    authToken = user.token
    testProduct = await createTestProduct({ price: 1000, stock: 10 })
  })
 
  afterEach(async () => {
    await clearTestData()
  })
 
  it("should create order successfully", async () => {
    const res = await request(app)
      .post("/api/orders")
      .set("Authorization", `Bearer ${authToken}`)
      .send({ productId: testProduct.id, quantity: 2 })
 
    expect(res.status).toBe(201)
    expect(res.body.order.total).toBe(2000)
 
    // verify inventory updated
    const product = await db.product.findUnique({ where: { id: testProduct.id } })
    expect(product?.stock).toBe(8) // 10 - 2
  })
 
  it("should return 400 when out of stock", async () => {
    const res = await request(app)
      .post("/api/orders")
      .set("Authorization", `Bearer ${authToken}`)
      .send({ productId: testProduct.id, quantity: 100 }) // more than stock
 
    expect(res.status).toBe(400)
    expect(res.body.error.code).toBe("E_OUT_OF_STOCK")
  })
})

E2E testing — exercising the critical user journey

E2E tests use Playwright to imitate a real user. Focus only on the journeys whose failure would hurt revenue most:

> Build E2E tests with Playwright for
  3 critical user journeys:
 
  Journey 1: Purchase flow (the most important)
  - Open the homepage → pick a product → add to cart
  - → checkout → fill in shipping → choose payment
  - → confirm the order → see the confirmation page
  - → receive the confirmation email
 
  Journey 2: User registration + first purchase
  - Fill in email, password, verify the email
  - → make the first purchase → see the welcome discount
 
  Journey 3: Search + filter + purchase
  - Search for a product → filter by category and price
  - → pick a product → buy it
 
  Setup:
  - Use a separate test user account
  - Run against the staging environment
  - Screenshot on failure
  - Retry twice automatically

// tests/e2e/purchase-flow.spec.ts
import { test, expect } from "@playwright/test"
 
test.describe("Purchase Flow", () => {
  test.beforeEach(async ({ page }) => {
    // log in with the test account
    await page.goto("/login")
    await page.fill("[name=email]", process.env.TEST_USER_EMAIL!)
    await page.fill("[name=password]", process.env.TEST_USER_PASSWORD!)
    await page.click("[type=submit]")
    await page.waitForURL("/dashboard")
  })
 
  test("complete purchase flow", async ({ page }) => {
    // pick a product
    await page.goto("/products")
    await page.click("[data-testid=product-card]:first-child")
    await page.click("[data-testid=add-to-cart]")
 
    // Checkout
    await page.goto("/cart")
    await page.click("[data-testid=checkout-btn]")
 
    // fill in shipping
    await page.fill("[name=address]", "123 Test Street")
    await page.fill("[name=city]", "Bangkok")
    await page.selectOption("[name=province]", "BKK")
    await page.click("[data-testid=next-btn]")
 
    // choose payment (test card)
    await page.fill("[name=card-number]", "4242424242424242")
    await page.fill("[name=expiry]", "12/28")
    await page.fill("[name=cvc]", "123")
    await page.click("[data-testid=pay-btn]")
 
    // verify confirmation
    await expect(page.getByText("Order Confirmed")).toBeVisible()
    await expect(page.getByTestId("order-id")).toContainText("ORD-")
  })
})
 
// playwright.config.ts
export default {
  retries: 2,
  screenshot: "only-on-failure",
  video: "retain-on-failure",
  use: { baseURL: process.env.STAGING_URL },
}

The test coverage report — measuring completeness

# view the coverage report
npx vitest run --coverage
 
# have Claude analyse the coverage gaps
> Here is the coverage report:
  File: src/lib/pricing.ts — 67% coverage
  Uncovered lines: 45-67, 89-102
 
  Read @src/lib/pricing.ts lines 45-67 and 89-102
  and write the missing test cases,
  especially for the error handling paths
 
# set a minimum coverage threshold
// vitest.config.ts
coverage: {
  thresholds: {
    lines: 80,
    functions: 80,
    branches: 70,
  }
}

Phase 3: Security audit — defending before you are attacked

The security audit is the step vibe coders skip most, on the assumption that Claude probably wrote safe code. The truth is that Claude can miss a security issue just as easily.

A security audit checklist with Claude

# .claude/commands/security-audit.md
# Security Audit
 
Run a comprehensive security audit on this PR:
 
## OWASP Top 10 Checklist
 
### 1. Injection
- SQL injection: are parameterised queries or an ORM used?
- NoSQL injection: is input validated before the query?
- Command injection: is user input kept out of the shell?
 
### 2. Broken Authentication
- Is a session timeout configured?
- Are stored passwords hashed with bcrypt/argon2?
- Is there brute force protection?
- JWT: is the expiry sensible and the signature verified correctly?
 
### 3. Sensitive Data Exposure
- HTTPS on every endpoint?
- Is sensitive data in the logs?
- Passwords/tokens in an error message?
- Does the API response include fields it should not?
 
### 4. Access Control
- Does every endpoint have an auth check?
- Can user A reach user B's data? (IDOR)
- Do admin-only routes have a role check?
 
### 5. XSS
- Is user input rendered as HTML sanitised?
- Is dangerouslySetInnerHTML used with user input?
- Is a Content-Security-Policy header set?
 
### 6. CSRF
- Do form submissions carry a CSRF token?
- Do state-changing requests use POST rather than GET?
 
## Report
### 🔴 Critical (fix immediately):
### 🟠 High (fix before production):
### 🟡 Medium (fix in this sprint):
### 🟢 Low (fix later):

Examples of finding and fixing security issues

IDOR (insecure direct object reference)

❌ Vulnerable to IDOR // GET /api/orders/:id const order = await db.order.findUnique({ where: { id: params.id } }) // no check that the order belongs to this user! // user A can view user B’s order	✅ IDOR prevented const order = await db.order.findFirst({ where: { id: params.id, userId: session.user.id // check ownership } }) if (!order) { throw new NotFoundError() // do not reveal that it exists but is off-limits }

Mass assignment

❌ Vulnerable // the user can send any body they like await db.user.update({ where: { id: userId }, data: req.body // dangerous! }) // a user could send { role: “admin” }	✅ Whitelist the fields const { name, email, bio } = req.body // accept only the permitted fields await db.user.update({ where: { id: userId }, data: { name, email, bio } }) // role cannot be changed

Environment variable security

> Audit the security of the environment variables:
  1. Is there a .env.example with no real secrets in it?
  2. Is .env in .gitignore?
  3. Is there any console.log of env vars?
  4. Is there validation that the required env vars exist at startup?
 
  Create src/lib/env.ts that:
  - validates every env var when the app starts
  - throws an error if a required var is missing
  - gives type-safe env vars via zod
 
// src/lib/env.ts
import { z } from "zod"
 
const envSchema = z.object({
  DATABASE_URL: z.string().url(),
  NEXTAUTH_SECRET: z.string().min(32),
  STRIPE_SECRET_KEY: z.string().startsWith("sk_"),
  NEXT_PUBLIC_APP_URL: z.string().url(),
})
 
export const env = envSchema.parse(process.env)
// throws at startup if the env is not valid

Phase 4: Performance benchmark — measure before you tune

You should never optimise anything without numbers. “Slow” and “fast” have to be defined precisely first, and only then do you benchmark and compare.

Performance budget — setting the target

A performance budget is the number everyone agrees means “good enough”. Before optimising you have to know what the target is:

Metric	Poor	Needs Work	Good	Excellent
Page Load (LCP)	> 4s	2.5-4s	< 2.5s	< 1s
API Response (P95)	> 1000ms	500-1000ms	< 500ms	< 100ms
Time to Interactive	> 5s	3.8-5s	< 3.8s	< 1s
First Byte (TTFB)	> 1800ms	900-1800ms	< 900ms	< 200ms
Core Web Vitals (CLS)	> 0.25	0.1-0.25	< 0.1	< 0.05

# add to CLAUDE.md
## Performance Budget
- API response P95: < 500ms (target < 200ms)
- Page load LCP: < 2.5s on 4G mobile
- Database query: < 100ms for every query
- Memory usage: < 512MB per instance
 
## Anything that exceeds the budget needs approval first

Database performance benchmark

> Build a performance benchmark for
  the most frequently used database queries:
 
  1. Run EXPLAIN ANALYZE for every important query
  2. Measure execution time against dataset sizes of:
     - 1,000 rows (current dev)
     - 100,000 rows (3 months out)
     - 1,000,000 rows (1 year out)
  3. Identify the queries that will be slow at large scale
  4. Propose the indexes that need adding
 
# build the benchmark script
// benchmark/db-queries.ts
import { db } from "../src/lib/db"
import { performance } from "perf_hooks"
 
async function benchmark(name: string, fn: () => Promise<unknown>) {
  const runs = 100
  const times: number[] = []
 
  for (let i = 0; i < runs; i++) {
    const start = performance.now()
    await fn()
    times.push(performance.now() - start)
  }
 
  times.sort((a, b) => a - b)
  console.log(`${name}:`)
  console.log(`  P50: ${times[50].toFixed(2)}ms`)
  console.log(`  P95: ${times[95].toFixed(2)}ms`)
  console.log(`  P99: ${times[99].toFixed(2)}ms`)
}
 
await benchmark("getProductList", () =>
  db.product.findMany({ take: 20, include: { category: true } })
)
 
await benchmark("searchProducts", () =>
  db.product.findMany({ where: { name: { contains: "test" } }, take: 20 })
)

API load testing with k6

Load testing simulates many users at once, which exposes bottlenecks before the real traffic arrives:

> Build a load test script with k6 for:
  Scenario 1: Normal load
  - 50 concurrent users, 5 minutes
  - Actions: browse products, search, view detail
 
  Scenario 2: Peak load
  - Ramp from 0 → 200 users over 2 minutes
  - Hold 200 users for 5 minutes
  - Ramp down over 2 minutes
 
  Scenario 3: Stress test
  - Find the breaking point: at how many users does the server fall over?
 
  Success criteria:
  - Error rate < 1%
  - P95 response < 500ms
  - No memory leak (memory does not keep climbing)

// load-test.js (k6 script)
import http from "k6/http"
import { check, sleep } from "k6"
import { Rate } from "k6/metrics"
 
const errorRate = new Rate("errors")
 
export const options = {
  stages: [
    { duration: "2m", target: 50 },   // Ramp up
    { duration: "5m", target: 50 },   // Stay at 50
    { duration: "2m", target: 200 },  // Ramp to peak
    { duration: "5m", target: 200 },  // Stay at peak
    { duration: "2m", target: 0 },    // Ramp down
  ],
  thresholds: {
    http_req_duration: ["p(95)<500"],  // P95 < 500ms
    errors: ["rate<0.01"],             // Error < 1%
  },
}
 
const BASE_URL = "https://staging.myapp.com"
 
export default function() {
  // Browse products
  const res = http.get(`${BASE_URL}/api/products?page=1`)
  errorRate.add(res.status !== 200)
  check(res, { "products loaded": (r) => r.status === 200 })
  sleep(1)
 
  // Search
  const search = http.get(`${BASE_URL}/api/products?q=phone`)
  errorRate.add(search.status !== 200)
  sleep(0.5)
}
 
# run with: k6 run load-test.js
# view the result: k6 run --out json=result.json load-test.js

Frontend performance — Core Web Vitals

> Analyse and improve the Core Web Vitals:
 
  1. Run Lighthouse CI against staging:
     npx lhci autorun
 
  2. Measure the current metrics:
     - LCP (Largest Contentful Paint)
     - FID / INP (Interaction to Next Paint)
     - CLS (Cumulative Layout Shift)
 
  3. Have Claude analyse the report:
> Here is the Lighthouse report:
  [paste JSON output]
  Identify the top 5 improvements with the highest impact,
  ordered by effort vs impact
 
  4. The improvements that come up most often:
     - Image optimisation (next/image)
     - Font loading (display: swap)
     - Unused JavaScript (tree shaking)
     - Server-side rendering where it fits

# lighthouserc.js
module.exports = {
  ci: {
    collect: {
      url: ["https://staging.myapp.com", "/products", "/checkout"],
      numberOfRuns: 3,
    },
    assert: {
      assertions: {
        "categories:performance": ["error", { minScore: 0.8 }],
        "categories:accessibility": ["error", { minScore: 0.9 }],
        "first-contentful-paint": ["error", { maxNumericValue: 2000 }],
        "largest-contentful-paint": ["error", { maxNumericValue: 2500 }],
        "cumulative-layout-shift": ["error", { maxNumericValue: 0.1 }],
      },
    },
  },
}

Phase 5: QA testing — before it reaches the user

QA (quality assurance) is the stage where a feature is tested from a real user's point of view, not merely for technical correctness. Claude can build a test plan that covers the ground.

Building a QA test plan with Claude

> Build a QA test plan for the feature: User Profile Edit
 
  Feature requirements:
  - The user can edit name, email, bio and avatar
  - The email must be re-verified when changed
  - The avatar is resized automatically to max 500x500px
  - A preview is shown before saving
 
  Produce a test plan covering:
  1. Functional tests: every feature works
  2. Negative tests: what happens on bad input
  3. Boundary tests: the limit values
  4. UX tests: the user experience flow
  5. Accessibility tests: keyboard, screen reader
  6. Cross-browser: Chrome, Firefox, Safari
  7. Mobile responsive: iPhone, Android
 
  Format: Test ID | Test Case | Steps | Expected | Priority

An example QA test plan

Test ID	Test Case	Steps	Expected	Priority
QA-001	Name edited successfully	1. Enter a new name 2. Save	The name changes immediately	High
QA-002	Empty name	1. Delete the name 2. Save	Error: Name required	High
QA-003	Name too long	1. Enter 200 characters 2. Save	Error: Max 100 chars	Medium
QA-004	New email	1. Change the email 2. Save	A verification email is sent	High
QA-005	Oversized avatar	1. Upload a 5MB PNG	Auto resize < 500KB	Medium
QA-006	Wrong avatar format	1. Upload a .pdf	Error: Invalid format	Medium
QA-007	Preview before save	1. Change the picture 2. View the preview	The preview is correct	Low

Regression testing — keeping old features from breaking

When you deploy a new feature you need confidence that the old ones still work. Claude can build the regression test suite:

> Build a regression test suite for this release
  to check that the older features still work:
 
  Critical features to test:
  - User login/logout
  - Product listing and search
  - Add to cart
  - Checkout flow
  - Order history
  - Payment
 
  Write a Playwright script that exercises every feature
  in 15 minutes (fast enough for CI/CD)
  against the staging environment
 
# schedule it to run automatically before a production deploy
# if it fails → block the deployment

Accessibility testing — usable by everyone

> Audit the accessibility of @src/components/
  against WCAG 2.1 Level AA:
 
  1. Semantic HTML: are heading, button and nav used correctly?
  2. Keyboard navigation: does Tab reach every interactive element?
  3. Focus indicator: is it obvious where the focus is?
  4. Alt text: does every image have a meaningful alt?
  5. Colour contrast: does it pass the 4.5:1 ratio?
  6. Error messages: can a screen reader read them?
  7. Form labels: does every input have a label?
 
  Write axe-core tests for the important pages
 
// accessibility.test.ts
import { test, expect } from "@playwright/test"
import AxeBuilder from "@axe-core/playwright"
 
test("checkout page passes accessibility", async ({ page }) => {
  await page.goto("/checkout")
  const results = await new AxeBuilder({ page }).analyze()
  expect(results.violations).toEqual([])
})

Phase 6: Deployment strategy — releasing safely

Deploying with confidence requires a strategy that lowers the risk and lets you roll back quickly when something goes wrong.

Environment strategy

Environment	Used for	Database	Deployed when
Local dev	Individual development	Local SQLite/PG	Constantly
Staging	QA + integration tests	Staging DB (copy of prod)	Every PR merge
Preview	Per-PR review	Shared with staging	Every PR created
Production	Real users	Production DB	Manual approval

The deployment checklist before production

# .claude/commands/deploy-check.md
# Pre-Production Deployment Checklist
 
Check every item before approving a production deploy:
 
## Code Quality
□ All tests pass (unit + integration + e2e)
□ No TypeScript errors
□ No ESLint errors
□ Code review approved
□ Security audit passed
 
## Database
□ Migration already tested on staging
□ Migration reversible (a rollback script exists)
□ Is there a recent data backup?
□ Index strategy reviewed
 
## Configuration
□ Environment variables complete in production
□ API keys correct (not test keys)
□ Rate limits configured
□ CORS whitelist configured correctly
 
## Monitoring
□ Error tracking (Sentry) connected
□ Performance monitoring connected
□ Alert rules configured
□ Dashboard ready to watch
 
## Rollback Plan
□ A way to roll the code back in < 5 minutes
□ A way to roll the database back if the migration breaks
□ Who is on call for production during the deploy
 
Run: git diff main...HEAD --stat and summarise it for me

Blue-green deployment

Blue-green means keeping two identical environments and switching traffic when you deploy, which makes rollback instant and downtime-free:

> Explain the blue-green deployment strategy
  and build a deployment script for:
  Infrastructure: Vercel (or AWS ECS)
 
  Flow:
  1. Deploy the new version to the Green environment
  2. Run smoke tests on Green
  3. If they pass → switch traffic from Blue → Green
  4. If they fail → stay on Blue, debug Green
  5. Once stable → Green becomes Blue for the next deploy
 
# deploy.sh
#!/bin/bash
set -e
 
echo "🚀 Starting deployment..."
 
# Deploy to staging
vercel --prod --scope=team --yes
 
# Run smoke tests
echo "🔍 Running smoke tests..."
npx playwright test tests/smoke/ --project=chromium
 
if [ $? -eq 0 ]; then
  echo "✅ Deployment successful!"
  # Notify Slack
  curl -X POST $SLACK_WEBHOOK -d '{"text":"✅ Deploy success!"}'
else
  echo "❌ Smoke tests failed! Rolling back..."
  vercel rollback
  curl -X POST $SLACK_WEBHOOK -d '{"text":"❌ Deploy failed, rolling back"}'
  exit 1
fi

Canary release — letting it out a little at a time

A canary release ships the feature to a fraction of users first and expands only if nothing goes wrong. It suits large, high-risk features:

// Feature flags for a canary release
// src/lib/feature-flags.ts
import { getServerSession } from "next-auth"
 
export async function isFeatureEnabled(flag: string): Promise<boolean> {
  // read from LaunchDarkly, Unleash, or the database
  const flags = await getFeatureFlags()
  return flags[flag]?.enabled ?? false
}
 
// use it in a component
const showNewCheckout = await isFeatureEnabled("new_checkout_v2")
 
return showNewCheckout ? <NewCheckout /> : <OldCheckout />
 
# Canary rollout plan
# Day 1: 5% of users
# Day 3: 25% if nothing has gone wrong
# Day 7: 50%
# Day 10: 100%

Phase 7: Monitoring — knowing before the user tells you

Good monitoring means you learn the system has a problem before a user complains. You can build it in an hour or two with Claude.

Error tracking with Sentry

> Set up Sentry for this Next.js app:
 
  1. Create sentry.client.config.ts and sentry.server.config.ts
  2. Capture every uncaught error
  3. Enrich errors with: user ID, session info,
     request details, environment
  4. Ignore the errors that do not matter: network errors from bots
  5. Set up alerts: email when the error rate goes above 1%
  6. Add release tracking so we can see which deploy caused an error
 
// sentry.server.config.ts
import * as Sentry from "@sentry/nextjs"
 
Sentry.init({
  dsn: process.env.SENTRY_DSN,
  environment: process.env.NODE_ENV,
  tracesSampleRate: process.env.NODE_ENV === "production" ? 0.1 : 1.0,
  beforeSend(event) {
    // do not send errors from bots
    if (event.request?.headers?.["user-agent"]?.includes("bot")) {
      return null
    }
    return event
  }
})

Application metrics

// src/lib/metrics.ts — custom metrics
import { register, Counter, Histogram } from "prom-client"
 
export const httpRequestDuration = new Histogram({
  name: "http_request_duration_seconds",
  help: "Duration of HTTP requests in seconds",
  labelNames: ["method", "route", "status"],
  buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5],
})
 
export const orderCreated = new Counter({
  name: "orders_created_total",
  help: "Total number of orders created",
  labelNames: ["payment_method"],
})
 
export const paymentFailed = new Counter({
  name: "payment_failures_total",
  help: "Total number of payment failures",
  labelNames: ["reason"],
})
 
// usage
orderCreated.inc({ payment_method: "credit_card" })
paymentFailed.inc({ reason: "insufficient_funds" })
 
// the dashboard reads from /metrics

Alert rules — automatic notification

> Build the alert rules for production:
 
  Critical (PagerDuty + SMS):
  - Error rate > 5% over 5 minutes
  - API P95 > 2000ms over 10 minutes
  - Payment failure rate > 10%
  - Server CPU > 90% for more than 5 minutes
 
  Warning (Slack):
  - Error rate > 1%
  - API P95 > 500ms
  - Database connection pool > 80%
  - Memory > 80%
 
  Info (log only):
  - New user registration
  - Order created
  - Deploy completed
 
# build the Slack alert function
async function sendAlert(level: string, message: string) {
  await fetch(process.env.SLACK_WEBHOOK!, {
    method: "POST",
    body: JSON.stringify({
      text: `${level === "critical" ? "🚨" : "⚠️"} ${message}`,
    })
  })
}

Phase 8: A/B testing — measuring with real data

A/B testing runs two versions at once to measure which is better from real user behaviour, rather than from the team’s opinion.

When should you run an A/B test?

Worth an A/B test	Not worth an A/B test
UI/UX expected to affect conversion	Bug fixes (clearly need fixing)
Pricing or promotion strategy	Security improvements
A new onboarding flow	Features everyone is asking for
CTA copy or button design	Technical refactoring
An improved checkout flow	Compliance requirements
A changed search algorithm	Too small a user base (< 1000/day)

Setting up an A/B test with Claude

> Build an A/B test framework for the checkout button:
 
  Hypothesis:
  "Changing the CTA from Checkout to Buy Now will raise conversion"
 
  Primary metric: conversion rate (cart → order)
  Secondary metrics: revenue per user, time to checkout
  Guard rails: cart abandonment must not increase
 
  Sample size: calculate it for statistical significance
  Duration: 2 weeks (to cover the full weekly pattern)
 
  Produce:
  1. The experiment config
  2. The assignment logic (which user sees which variant)
  3. Event tracking
  4. The analysis query

// src/lib/experiments.ts
import { db } from "./db"
import { hashUserId } from "./utils"
 
interface Experiment {
  name: string
  variants: string[]
  weights: number[]  // must add up to 100
  startDate: Date
  endDate: Date
}
 
export async function getVariant(
  userId: string,
  experiment: Experiment
): Promise<string> {
  // Deterministic assignment: the same user gets the same variant every time
  const hash = hashUserId(userId + experiment.name)
  const bucket = hash % 100
 
  let cumulative = 0
  for (let i = 0; i < experiment.variants.length; i++) {
    cumulative += experiment.weights[i]
    if (bucket < cumulative) {
      // Log assignment
      await logExperimentAssignment(userId, experiment.name, experiment.variants[i])
      return experiment.variants[i]
    }
  }
  return experiment.variants[0]
}
 
// usage
const checkoutExperiment: Experiment = {
  name: "checkout_cta_q1_2025",
  variants: ["control", "buy_now"],
  weights: [50, 50],  // 50/50 split
  startDate: new Date("2025-01-15"),
  endDate: new Date("2025-01-29"),
}
 
const variant = await getVariant(session.user.id, checkoutExperiment)
const buttonText = variant === "buy_now" ? "Buy Now" : "Checkout"

Event tracking for the A/B test

// src/lib/analytics.ts
export async function trackEvent(
  userId: string,
  event: string,
  properties: Record<string, unknown>
) {
  await db.analyticsEvent.create({
    data: {
      userId,
      event,
      properties: JSON.stringify(properties),
      timestamp: new Date(),
    }
  })
}
 
// the important tracking calls in the checkout flow
await trackEvent(userId, "checkout_started", {
  experiment: "checkout_cta_q1_2025",
  variant: variant,
  cartValue: cart.total,
})
 
await trackEvent(userId, "order_completed", {
  experiment: "checkout_cta_q1_2025",
  variant: variant,
  orderId: order.id,
  revenue: order.total,
})

Analysing the A/B test result

> Analyse the result of this A/B test:
 
  Experiment: checkout_cta_q1_2025
  Duration: 2025-01-15 to 2025-01-29
 
  Control (Checkout):
  - Impressions: 5,234
  - Conversions: 523 (10.0% conversion rate)
  - Revenue: ฿2,615,000
 
  Variant (Buy Now):
  - Impressions: 5,198
  - Conversions: 571 (10.98% conversion rate)
  - Revenue: ฿2,882,000
 
  Tell me:
  1. Statistical significance (p-value)?
  2. Confidence interval?
  3. Effect size?
  4. Should we ship the variant, and why?
  5. What you would recommend for the next experiment
 
# the analysis SQL query
SELECT
  properties->>variant AS variant,
  COUNT(DISTINCT CASE WHEN event = "checkout_started" THEN userId END) as impressions,
  COUNT(DISTINCT CASE WHEN event = "order_completed" THEN userId END) as conversions,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN event = "order_completed" THEN userId END)
    / COUNT(DISTINCT CASE WHEN event = "checkout_started" THEN userId END), 2) AS conversion_rate,
  SUM(CASE WHEN event = "order_completed" THEN (properties->>revenue)::numeric END) AS revenue
FROM analytics_events
WHERE properties->>experiment = "checkout_cta_q1_2025"
  AND timestamp BETWEEN "2025-01-15" AND "2025-01-29"
GROUP BY variant

Building on chapter 4: advanced loop debugging

Chapter 4 gave you the basic protocol. This section goes deeper into techniques that work on production bugs, which are more complicated.

The root cause analysis (RCA) framework

When production has a serious problem, use RCA to find the true root cause rather than merely patching the symptom:

# .claude/commands/rca.md
# Root Cause Analysis
 
Run an RCA for this incident:
 
## Timeline
Lay out what happened, in order:
- [time] what happened
- [time] who found it
- [time] what was done about it
- [time] back to normal
 
## 5 Whys Analysis
Ask "why" five times over, until you reach the root cause:
Why? → Why? → Why? → Why? → Why?
 
## Root Causes
- Proximate cause: what directly triggered the incident
- Root cause: the true origin
- Contributing factors: what made it worse
 
## Impact
- Users affected: [count]
- Duration: [time]
- Revenue impact: [estimate]
 
## Action Items
| Action | Owner | Priority | Due Date |
|--------|-------|----------|----------|
| [fix the root cause] | [name] | P0 | [date] |
| [prevent a repeat] | [name] | P1 | [date] |
 
## Preventing recurrence
What has to change in the process, the code, or the monitoring?

The production debugging protocol

When production breaks and the fix has to be fast, this protocol keeps your thinking systematic even under pressure:

1	Assess — take stock first Is it critical? How many users are affected? Revenue impact? Do we need to roll back right now?

> Production incident: [describe the problem]
  Here is the error log:
  [paste logs]
 
  Help me assess:
  1. Severity: P0/P1/P2/P3?
  2. Blast radius: what % of users are affected?
  3. Should we roll back immediately or fix forward?
  4. If we fix: how long will it take?
 
  Answer within 2 minutes; it does not have to be 100% complete
  A decision comes first

2	Contain — stop the damage Get the system stable first, even before the root cause is fixed

# the containment moves you will use most
# 1. Roll back the deployment
vercel rollback  # or git revert + deploy
 
# 2. Feature flag off the broken feature
await setFeatureFlag("new_payment", false)
 
# 3. Rate limiting to cut the load
# 4. Redirect traffic away from the broken endpoint
# 5. Scale up the servers if it is a load issue

3	Diagnose — find the root cause Analyse systematically instead of guessing

> Analyse this production incident:
 
  The logs we have:
  [paste all the logs]
 
  Timeline:
  - 14:30 Deployed version 2.4.1
  - 14:45 Error rate started climbing
  - 15:00 User complaints
 
  What changed in v2.4.1:
  [paste the git diff or PR description]
 
  Please:
  1. Analyse the logs for a pattern
  2. Connect the logs to the code that changed
  3. Identify the most likely root cause
  4. Propose further diagnostic steps if you are still unsure

4	Fix — repair and verify Fix the root cause, with tests, before deploying

5	Document — write everything down Produce a post-mortem so it does not repeat

Handling dependencies that cause loops

Loops often come from a dependency with a bug or a breaking change, not from our own code:

> This error appeared after updating packages:
  TypeError: Cannot read property "x" of undefined
  in node_modules/some-library/index.js
 
  npm list | grep some-library
  some-library@3.0.0  ← updated from 2.x
 
  Please:
  1. Work out where the breaking change is
  2. Read the library's changelog for what has to be done
  3. Propose a migration path
  4. If the migration is complicated, propose a downgrade
     with the version locked for stability
 
# Lock the version to stop auto-updates
# package.json
"some-library": "2.8.1"  # use the exact version
# not "^2.8.1" (allows minor updates)
# not "~2.8.1" (allows patch updates)

Debugging with time travel (git bisect automation)

When you do not know which commit introduced the bug, automate git bisect:

# build an automatic bisect script
# bisect-test.sh
#!/bin/bash
# this script is run by git bisect
# exit 0 = good commit
# exit 1 = bad commit
 
npm run build 2>/dev/null  # the build must succeed
if [ $? -ne 0 ]; then exit 1; fi
 
# run the specific test related to the bug
npx vitest run tests/pricing.test.ts 2>/dev/null
exit $?
 
# run the bisect
git bisect start
git bisect bad HEAD
git bisect good v2.3.0  # a version known to be good
git bisect run ./bisect-test.sh
 
# Git finds the commit that introduced the bug on its own
# then hand it to Claude to analyse
> Here is the bad commit:
  [paste commit diff]
  Explain how this commit causes the bug
  and what the correct fix is

Memory leak detection

A memory leak slows the server down bit by bit until it has to be restarted. It is a common production bug and a hard one to find:

> I suspect a memory leak in @src/server/
  Memory climbs from 200MB to 800MB over 24 hours,
  then a restart brings it back down to 200MB
 
  Please build:
  1. A memory monitoring endpoint: GET /api/debug/memory
     showing heap used, heap total, RSS
  2. A leak detection script that runs
     every minute and logs when memory grows > 10MB/minute
  3. Suspect code analysis:
     read @src/server/ for the patterns that usually leak memory:
     - EventEmitter without removeListener
     - setTimeout/setInterval without clearTimeout
     - Closures holding a reference
     - A cache with no size limit
     - Streams that are never closed
 
// memory-monitor.ts
setInterval(() => {
  const mem = process.memoryUsage()
  console.log({
    heapUsed: Math.round(mem.heapUsed / 1024 / 1024) + "MB",
    heapTotal: Math.round(mem.heapTotal / 1024 / 1024) + "MB",
    rss: Math.round(mem.rss / 1024 / 1024) + "MB",
    timestamp: new Date().toISOString()
  })
}, 60000)

A worked example: a feature from vibe coding to real production

This example walks the “coupon system” feature from the first line to production and an A/B test, so you can see the full pipeline in practice.

	Day 1: Vibe coding

> Build a coupon system:
  - Table: coupons (code, type, value, max_uses, expires_at)
  - Validate coupon: check code, expiry, usage limit
  - Apply coupon: reduce the price by type (percent/fixed)
  - Track usage: record which user redeemed it
 
  Tech: Next.js + Prisma + PostgreSQL
  Work step by step and wait for approval

	Day 2: Testing

# once the implementation is done
> Write comprehensive tests:
  - Unit: validateCoupon, calculateDiscount
  - Integration: POST /api/coupons/apply
  - Edge cases: expired, uses exhausted, duplicate coupon
 
> Write an E2E test:
  - User enters the coupon code → sees the reduced price → checks out
 
# run the tests
npm test
 
# if one fails, do not fix it yourself; send the error to Claude
> The failing test is:
  [paste error]
  Make it pass without changing the test logic

	Day 3: Security + performance

# Security audit
> /project:security-audit
 
# Performance benchmark
> Benchmark the validateCoupon function:
  - 100 consecutive calls
  - 50 concurrent calls
  - Database query analysis
 
# found: the lookup query takes 200ms
> Add an index on coupons.code
  and on coupon_usages.coupon_id + user_id
 
# benchmark again with the index
# result: 8ms (25x faster)

	Day 4: QA + staging deploy

# build the QA test plan
> Build a test plan for the coupon feature
  covering functional, negative, boundary and UX
 
# Deploy to staging
git push origin feature/coupon-system
# Vercel creates the preview URL automatically
 
# run the regression tests against staging
npx playwright test tests/e2e/ --project=chromium \
  --config=playwright.staging.config.ts
 
# Deploy check
> /project:deploy-check

	Day 5: Production + monitoring

# Canary deploy: 10% of users first
# set the feature flag: coupon_system = 10%
 
# Monitor for 24 hours
# - Error rate normal
# - Coupon apply success rate > 99%
# - No double-apply bugs
 
# if it holds → 100% of users
# set the feature flag: coupon_system = 100%

	Day 14: A/B test results

# after the coupon system has been stable for a week
# A/B test: the "first-time coupon" popup
 
> Analyse the A/B test result:
  Control: no popup
  - Conversion rate: 8.2%
  - Average order value: ฿850
 
  Variant: a popup giving new users a 10% coupon
  - Conversion rate: 11.5% (+40%!)
  - Average order value: ฿780 (-8%)
  - Net revenue per user: +29%
 
  p-value: 0.001 (statistically significant)
  Confidence: 99.9%
 
  Decision: ship the variant to 100%
  Next test: try a 15% coupon against the 10%

Glossary for this chapter

Quality gate	The quality bar code must clear before it can move to the next stage
Testing pyramid	The principle of proportioning tests: unit (70%), integration (20%), E2E (10%)
Integration test	A test of several components working together, the real database included
E2E test	End-to-end: testing the whole system through a real browser, like a real user
Core Web Vitals	The set of metrics Google uses to measure UX: LCP, INP, CLS
Load testing	Testing a system with heavy traffic to find the bottleneck
k6	A popular code-first load testing tool
OWASP Top 10	The list of the ten most common web security vulnerabilities
IDOR	Insecure direct object reference — the hole that lets a user reach someone else’s data
Blue-green deploy	A deployment strategy with two environments to switch between, giving instant rollback
Canary release	Shipping a feature to a fraction of users first, then expanding if nothing goes wrong
Feature flag	A switch that turns a feature on or off without deploying new code
A/B testing	Running 2 versions at once to measure which is better from real data
Statistical significance	The statistical confidence that the observed result is not luck
P95/P99	The 95th/99th percentile: the response time for the slowest 5%/1% of requests
Root cause analysis (RCA)	The process of finding the real “why” rather than merely patching the symptom
Post-mortem	The document analysing an incident after the fact, to prevent a repeat
Regression test	A test that old features still work after a new deploy
Memory leak	A program consuming ever more memory without giving it back