Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of Testing Methodology?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/reliant-labs-testing-methodology)
---
name: testing-methodology
description: Testing methodology — test harness design, mock strategies, flakiness prevention, and structured test planning
---
# Testing Methodology
## Phase 1: Test Infrastructure Discovery
Before writing any tests, understand what exists:
### Test Framework Detection
- What test frameworks are installed? (jest, pytest, go test, etc.)
- How are tests run? (`npm test`, `make test`, `go test ./...`)
- Where do tests live? (`__tests__/`, `*_test.go`, `tests/`)
- What's the naming convention? (`*.test.ts`, `*_test.py`, `Test*.java`)
### Existing Patterns
- What test utilities/helpers exist?
- Are there existing fixtures or factories?
- What mocking libraries are used?
- How is test data managed?
- Are there existing harnesses to build on?
### Test Execution Environment
- How is CI configured for tests?
- What's the coverage tool and current coverage?
- Are there test databases or containers?
- What environment variables do tests need?
**Document findings** - this information helps you and future agents write consistent tests.
## Phase 2: Test Harness Design
The key to testing complex systems is good harnesses. A harness handles setup, teardown, and provides utilities so individual tests stay focused.
### What to Mock vs What to Keep Real
**Keep REAL when:**
- It's the thing you're actually testing
- It's fast and deterministic (pure functions, in-memory DBs)
- Mocking it would miss real bugs (ORM behavior, query logic)
- The real thing is simpler than a mock
**MOCK when:**
- External services (APIs, third-party services)
- Non-deterministic sources (time, random, network)
- Slow resources that aren't under test
- Components with complex setup that aren't relevant to the test
**The boundary rule:** Mock at system boundaries, keep internals real.
### Harness Patterns
#### Integration Server Harness
For API/backend testing:
```
TestHarness
├── Server (real, started once per suite)
├── Database (isolated per test, migrations run)
├── External Service Mocks (controllable)
└── Test Client (authenticated, with helpers)
```
#### Database Harness
For data layer testing:
```
TestHarness
├── Database Connection (transaction per test, rolled back)
├── Migration Runner (schema always current)
├── Fixture Loader (predictable test data)
└── Query Assertion Helpers
```
#### Async/Queue Harness
For message processing:
```
TestHarness
├── Queue (in-memory or real, isolated)
├── Worker Pool (controllable concurrency)
├── Event Collector (capture for assertions)
└── Completion Waiter (with timeout)
```
#### External Service Harness
For integration points:
```
TestHarness
├── Mock Server (httptest, wiremock, etc.)
├── Request Recorder (verify calls made)
├── Response Programmer (control responses)
└── Failure Injector (test error handling)
```
## Phase 3: Writing Good Tests
### Test Structure
Every test should have:
1. **Arrange**: Set up preconditions (use harness)
2. **Act**: Execute the thing being tested
3. **Assert**: Verify the outcome
4. **Cleanup**: Handled by harness (teardown)
### Test Naming
Names should describe the scenario and expected outcome:
- `test_create_user_with_valid_email_succeeds`
- `TestPaymentProcessor_RefundExceedingBalance_ReturnsError`
- `it('should reject duplicate usernames', ...)`
### Avoiding Flakiness
**Timing issues:**
- Never use fixed sleeps (`time.Sleep(1s)`)
- Poll with timeout for async operations
- Use event-driven waiting when possible
```
// Bad
time.Sleep(2 * time.Second)
assert(result.IsComplete())
// Good
waitFor(func() bool { return result.IsComplete() }, timeout: 5s, poll: 100ms)
```
**State isolation:**
- Each test gets fresh state (new DB transaction, clean mocks)
- Don't rely on test execution order
- Clean up resources in teardown, not in test body
**Live / heavy tests are opt-in:**
- A test that hits a live external dependency (real sandbox, deployed environment, real credential) must be gated behind an explicit opt-in env var (e.g. `RUN_LIVE_TESTS=1`) and skip-with-a-reason when it's unset.
- The default suite stays hermetic and green in CI; the live test runs only when a human or a dedicated job asks for it. A live test that runs by default is a flake and a credential-leak risk.
**Non-determinism:**
- Mock time.Now() for time-dependent logic
- Seed random generators deterministically
- Control UUIDs/IDs in tests
### Test Categories
#### Unit Tests
- Test single functions/methods in isolation
- Mock all dependencies
- Fast (<10ms each)
- High coverage of edge cases
#### Integration Tests
- Test components working together
- Real database, real internal services
- Mock external boundaries
- Medium speed (<1s each)
#### E2E Tests
- Test complete user flows
- Minimal mocking (only external services)
- Slower but high confidence
- Focus on critical paths
#### Contract Tests
- Verify API contracts between services
- Both provider and consumer tests
- Catch breaking changes early
#### Concurrency Tests
- Test race conditions explicitly
- Use race detectors (go test -race)
- Stress test with goroutines/threads
- Test deadlock scenarios
## Phase 4: Test Tooling
### Run the cheapest tier that answers the question
Most of an edit loop's test time goes to compiling, linking, and re-running
tests whose result is already known — not to test bodies. Keep two tiers and
use the right one:
- **Inner loop, after each edit:** fast tests only, scoped to the packages you
touched, with the runner's cache on. In Go that is
`go test -short ./internal/<pkg>/...` with no `-count=1` and no `-race`.
Go's test cache keys on the test binary, its flags, and the files and env
vars the test reads, so a cached pass is a real pass for a hermetic test;
`-count=1` throws that away, and `-race` rebuilds everything under a
separate cache key and runs 2-10x slower. Never point the build cache at a
private directory — a cold cache recompiles the world.
- **Full gate, once before you finish (and in CI):** the whole suite with
`-race -count=1`, plus the tagged integration and e2e lanes.
- **Keep the fast tier fast.** A test that takes more than ~2s gets
`if testing.Short() { t.Skip("<why>") }`, or has its slow side-effect
bypassed under `-short`, and its slow path still runs in the full gate. Gate
it, don't gut it: never weaken an assertion to get under the budget.
- A run expected to take more than ~2 minutes goes in the background; never
block on a long foreground call.
<!-- @forge-only:start -->
In a forge project both tiers are Taskfile targets:
`task test:short -- ./internal/<pkg>/...` is the inner loop, and `task test`
is the full lane CI runs.
<!-- @forge-only:end -->
### Useful Test Scripts
Create tools that make testing easier. Write to `.reliant/tools/` or the project's scripts directory.
**Run the inner-loop tier (pass the packages you touched):**
```bash
#!/bin/bash
# .reliant/tools/test-unit.sh
go test -short "${@:-./...}"
```
**Run with coverage:**
```bash
#!/bin/bash
# .reliant/tools/test-coverage.sh
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out -o coverage.html
echo "Coverage report: coverage.html"
```
**Seed test database:**
```bash
#!/bin/bash
# .reliant/tools/seed-test-db.sh
DB_PATH="${TEST_DB:-./test.db}"
sqlite3 "$DB_PATH" < ./testdata/seed.sql
echo "Test database seeded at $DB_PATH"
```
**Run flaky test detector:**
```bash
#!/bin/bash
# .reliant/tools/flaky-detector.sh
TEST_NAME="$1"
RUNS="${2:-10}"
for i in $(seq 1 $RUNS); do
if ! go test -run "$TEST_NAME" ./...; then
echo "FLAKY: Failed on run $i"
exit 1
fi
done
echo "STABLE: Passed $RUNS consecutive runs"
```
## Phase 5: Test Plan Output
When reviewing code or planning tests, provide a structured test plan:
### Coverage Assessment
- What's currently tested
- What's missing coverage
- Risk areas without tests
### Unit Tests to Add
| Function/Method | Test Cases | Priority |
|-----------------|------------|----------|
| `validateEmail` | valid format, invalid format, empty, unicode | High |
| `calculateTotal` | normal, with discount, zero items, overflow | High |
### Integration Tests to Add
| Scenario | Components | Mocking Strategy |
|----------|------------|------------------|
| User registration flow | API + DB + Email | Mock email service |
| Payment processing | API + DB + Stripe | Mock Stripe API |
### E2E Tests to Add
| User Flow | Steps | Assertions |
|-----------|-------|------------|
| Complete checkout | Login -> Add item -> Pay -> Confirm | Order created, email sent |
### Concurrency Tests
- Race conditions to test
- Deadlock scenarios to verify
- Load/stress test considerations
### What to Mock vs Keep Real
| Component | Decision | Rationale |
|-----------|----------|-----------|
| Database | Real (postgres) | Need to test actual queries |
| Redis | Mock | Not testing caching logic |
| Stripe API | Mock | External, non-deterministic |
# Guidelines
## DO:
- Follow existing test patterns in the codebase
- Create reusable harnesses and helpers
- Test edge cases and error paths
- Make test failures self-explanatory
- Run tests before and after changes
## DON'T:
- Write tests that pass when code is broken
- Create flaky tests (fix or delete them)
- Over-mock (test realistic scenarios)
- Ignore existing test utilities
- Write tests without understanding the code
- Run the full suite with `-count=1` / `-race` after every edit — that is the full gate's job, not the inner loop's
## Test Quality Checklist
- [ ] Tests actually test the right thing (not just coverage theater)
- [ ] Tests fail when code is broken
- [ ] Tests are deterministic (no flakiness)
- [ ] Tests are fast enough for the feedback loop
- [ ] Tests have clear failure messages
- [ ] Test code is maintainable (DRY with helpers/fixtures)