Testing Methodology

---
name: testing-methodology
domain: dev
description: |
  [METHODOLOGY] Testing strategy (when to write which tests, test
  quality standards, test pyramid). Preloaded by dev-qa-engineer agent.
---

# Test Master

**Test Pyramid:**
```
        /\
       /E2E\        <- Few (3-5 critical flows)
      /------\
     /Integr.\      <- Some (all endpoints + DB)
    /----------\
   /   Unit     \   <- Many (all business logic)
  /--------------\
 /    Smoke      \  <- Minimal (1-2 basic tests)
/------------------\
```

---

## When to Use Each Test Type

### Smoke Tests
**Purpose:** Verify basic project setup works.

**Use for:**
- Testing framework is configured
- Environment variables accessible
- Basic imports work
- Infrastructure is functional

**Written:** During infrastructure setup (once per project)

**Setting up smoke tests?** Read [smoke-tests.md](references/smoke-tests.md) — CI integration, example templates.

---

### Unit Tests
**Purpose:** Test business logic in isolation.

**Use for:**
- Functions with calculations, validations, transformations
- Decision-making logic (if/else, switch)
- Data processing and formatting
- Error handling logic

**Written:** By code-developer during each task (immediately after code)

**Skip for:** Simple getters/setters, one-line changes, trivial updates

**Writing unit tests?** Read [unit-tests.md](references/unit-tests.md) — patterns, mocking, examples.

---

### Integration Tests
**Purpose:** Test API endpoints, database, and external services.

**Use for:**
- All API endpoints (POST/PUT/DELETE especially)
- Database operations (create/update/delete)
- External service integrations (payments, email, webhooks)

**Written:** As separate task at end of feature (if defined in Tech Spec)

**Rule:** Every API endpoint and every DB write operation must have a corresponding integration test. Missing integration tests for these is a quality gap.

**Writing integration tests?** Read [integration-tests.md](references/integration-tests.md) — API testing, DB setup, fixtures.

---

### E2E Tests
**Purpose:** Test critical user journeys end-to-end.

**Use for:**
- Top 3-5 most critical user flows
- Large features (>5 tasks)
- Critical business processes (auth, payment, core features)

**Written:** After deploy to dev, before manual testing (if proposed/requested)

**Writing E2E tests?** Read [e2e-tests.md](references/e2e-tests.md) — Playwright/Cypress setup, page objects, CI.

---

## Decision Framework

### Should I write unit tests for this?

**YES if:**
- Function has business logic
- Function makes decisions
- Function transforms data
- Function handles errors
- Task specifies testing

**NO if:**
- Simple getter/setter
- One-line text change
- Trivial config update
- No code written (research/docs)

### Should I write integration tests?

**YES if:**
- Tech Spec specifies integration tests
- Feature has API endpoints
- Feature interacts with database
- Feature calls external services

**NO if:**
- Tech Spec says "None"
- Feature is purely client-side
- Already covered by E2E tests

### Should I write E2E tests?

**YES if:**
- Feature has >5 tasks
- Feature touches critical flows
- Feature has breaking changes
- User explicitly requests
- Tech Spec specifies E2E tests

**NO if:**
- Small feature (<3 tasks)
- Non-critical functionality
- Well covered by unit + integration tests
- Time/cost constraints

---

## Key Testing Principles

1. **Write tests immediately** - In the same session as the code, before moving on
2. **Test behavior, not implementation** - Focus on what, not how
3. **Keep tests fast** - Unit: milliseconds, Integration: seconds, E2E: minutes
4. **Isolate tests** - Mock external dependencies in unit tests
5. **One concern per test** - Each test validates one thing
6. **Clear test names** - Describe what's tested and expected outcome
7. **Independent tests** - Each test runs with its own setup, no shared state
8. **Clean state** - Always start with known database state
9. **Tests must verify real behavior** - Assert on actual results, not mock calls
10. **Every test earns its place** - Each test catches a specific failure no other test catches (see below)

---

## Redundant Testing Anti-pattern

Tests that duplicate coverage waste time and create maintenance burden.

**Signs of redundant testing:**
- Same behavior verified by both unit test and integration test with no added value
- E2E test that only checks what unit tests already cover
- Multiple test files testing the same function with same scenarios
- "Tests for completeness" that exist without protecting against real regressions

**Rule:** Each test must justify its existence — it catches a specific failure that no other test catches. If removing the test reduces zero confidence, it belongs nowhere.

---

## Test Quality Requirements

### What Makes a BAD Test

**Tests nothing:**
```typescript
expect(true).toBe(true)
```

**Tests only that mock was called:**
```typescript
expect(api.call).toHaveBeenCalled()  // Without checking result
```

**No assertions:**
```typescript
render(<Component />)  // Just renders, checks nothing
```

### What Makes a GOOD Test

**Tests actual result:**
```typescript
expect(calculateTotal(100, 0.2)).toBe(80)
```

**Tests real state change:**
```typescript
cart.add({ id: 1 })
expect(cart.items).toHaveLength(1)
```

### Rule: Excessive Mocking = Wrong Test Type

If mocking 3+ dependencies -> use integration or E2E test instead.

---

## When to Prioritize E2E Over Unit Tests

For some project types, E2E tests are MORE valuable than unit tests:

| Project Type | Primary Tests | Why |
|--------------|---------------|-----|
| API/Backend | Unit + Integration | Logic in functions |
| CLI Tools | Unit + Integration | Testable in isolation |
| **UI Apps** | **E2E + Integration** | Logic in UI interaction |
| **Browser Extensions** | **E2E (real browser)** | APIs can't be mocked reliably |
| **Mobile Apps** | **E2E** | Platform APIs need real env |

**Rule:** If mocking more than testing -> wrong test type.

---

## Mocking Strategy

### Unit Tests
- **Mock:** Database, API calls, file system, time
- **Why:** Fast, isolated, deterministic
- **How:** Use framework mocking (jest.mock, unittest.mock)

### Integration Tests
- **Real:** Database (test DB), file system
- **Mock:** External services (payments, email)
- **Why:** Test real interactions, avoid external costs/delays

### E2E Tests
- **Real:** Everything (use test/sandbox mode for external services)
- **Why:** Test complete real-world scenario

---

## Test Quality Review

**When reviewing existing tests**, read [test-quality-review.md](references/test-quality-review.md) — categories of bad tests, severity levels, decision criteria.