Audit

---
name: audit
description: Run a multi-category code audit on the cqs codebase. Spawns parallel agents per batch.
disable-model-invocation: true
argument-hint: "[batch-number|all]"
---

# Audit

Run the 16-category code audit.

## Implementation — the `audit` workflow (preferred)

The discovery + synthesis + disposition phases now run as one reusable workflow, `.claude/workflows/audit.js`:

```
Workflow({ name: 'audit', args: { churn: '<recently-changed areas to scrutinize>' } })
```

- **Discovery** — all 16 categories in ONE parallel stage (auditors return structured findings; no team, no 2-batch split — the concurrency cap schedules them).
- **Synthesize** — one agent dedups/clusters, triages P1–P4 (refining the deterministic difficulty×impact first-pass), assigns each row a disposition `route`, and WRITES `docs/audit-findings.md` + `docs/audit-triage.md`.
- **Disposition** — per row: `auto-fix` → generate a fix-spec + adversarially verify it against source; `issue` → draft a title+body; `tracked`/`inline`/`drop` → skip. Gen/verify/draft agents are **read-only** (`Explore`) so prepare-mode can't mutate the tree. `args.implement=true` additionally runs worktree-isolated implementers on verified fixes.

**The orchestrator (main loop) keeps the gated acts** — deliberately NOT in the workflow because they're irreversible / outward-facing / serialized to protected `main`:
- Review the triage — the cornerstone P1-vs-P2 + accept/reject/route-override call.
- `/land` the verified fixes (serialized, CI-gated).
- `gh issue create` the drafts, after dedup against open issues.

Route by **fix-nature, not tier**: a high-impact finding whose fix is a judgment call (architecture/API/perf-tradeoff) is `issue`, never `auto-fix`.

The team-based process below is the legacy reference — kept for the category scopes and the fix-prompt-review detail the workflow embodies. Teams are currently unavailable, so use the workflow.

## Arguments

- `$ARGUMENTS` — batch number (1-2) or `all` for full audit

**`all` means run BOTH batches** — all 16 categories. Run batch 1, then batch 2, then triage all findings together. Do NOT stop after batch 1.

## Batches

There are 16 categories split into 2 batches of 8. Each batch spawns 8 parallel agents (one per category). `all` runs both batches sequentially.

| Batch | Categories |
|-------|-----------|
| 1 | Code Quality, Documentation, API Design, Error Handling, Observability, Test Coverage (adversarial), Robustness, Scaling & Hardcoded Limits |
| 2 | Algorithm Correctness, Extensibility, Platform Behavior, Security, Data Safety, Performance, Resource Management, Test Coverage (happy path) |

## Process

### Setup

1. **Archive previous audit**: If `docs/audit-findings.md` or `docs/audit-triage.md` exist, rename both with the version suffix (e.g., `audit-findings-v0.9.1.md`, `audit-triage-v0.9.1.md`). Each audit starts with fresh files.

2. **Enable audit mode**: `cqs audit-mode on --expires 2h` — prevents stale notes from biasing review (no `-q` flag exists; the subcommand takes only `on`/`off` + `--expires`)

### Per-Batch

3. **Create team**: One team per batch (`audit-batch-N`)

4. **Spawn teammates**: One per category. Use `model: "fable"` for every auditor (per the 2026-06-11 model split: review/audit-finder lanes → fable) — the 2026-06-10 review/fix campaign showed Fable at or above opus quality on exactly this kind of judgment work (15/15 review findings confirmed, premise-drift catches, empirical hypothesis correction). Sonnet produces lower-quality judgments and haiku misses subtle findings. The per-category scope keeps each agent focused enough that frontier-model cost is reasonable.

   **Exception — the Security category auditor uses `model: "opus"`.** Fable's documented bug-finding gains explicitly exclude security-focused analysis (its cyber classifiers apply there), and benign-adjacent security work can occasionally trigger a classifier false positive — a mid-run refusal kills that category's coverage for the whole audit. Opus carries no refusal risk on this lane, has no documented capability deficit for it, and costs half. Same reasoning applies to `/boundary-probe`.

   **Nested-lead option for broad categories** (Code Quality, both Test Coverage categories, Performance, Data Safety): spawn the teammate as `subagent_type: "general-purpose"` (it needs the Agent tool) and have it (1) run the category's mandatory cqs commands itself, (2) fan out 3 read-only sub-scope agents in parallel (`subagent_type: "explorer"`, omit model so they inherit; HARD RULE in their prompts: no file writes — return candidate findings as their final message), (3) verify every candidate against the cited source + archived triages before appending, and report the candidate→appended funnel. Measured v1.42: 38 candidates → 31 appended (1 cross-category dup caught, 5 same-root-cause merges, 1 stale-triage reject); the two nested categories produced the largest verified hauls and the most P1s. Narrow/mined-out categories (Scaling found 1 finding in v1.42) stay single-agent — the lead overhead isn't worth it.

5. **Each teammate prompt must include**:
   - Their category scope (from table below)
   - Instruction to read archived triage files (e.g., `docs/audit-triage-v*.md`) — skip anything already triaged in prior audits
   - Instruction to read `docs/audit-findings.md` first — skip anything already reported by earlier batches in this audit
   - Instruction to append findings to `docs/audit-findings.md` **via bash heredoc append** (`cat >> docs/audit-findings.md <<'EOF' ... EOF`) — never Edit/Write on that file; 8 agents append concurrently and Edit fails with "file modified since read". This replaces the old per-category scratch-file + aggregation step: v1.42 ran 8 concurrent heredoc appenders with zero conflicts. (Initialize the findings file with a header before spawning so appends have a base.)
   - Format: `## [Category]\n\n#### [Finding title]\n- **Difficulty:** easy | medium | hard\n- **Location:** ...\n- **Description:** ...\n- **Suggested fix:** ...`
   - Use `subagent_type: "auditor"` when spawning — the auditor agent definition (`.claude/agents/auditor.md`) has cqs tools built in

6. **Shutdown team** after all agents complete

### After All Batches

7. **Triage**: Read `docs/audit-findings.md` in full, then classify:
   - P1: Easy + high impact → fix immediately
   - P2: Medium effort + high impact → fix in batch
   - P3: Easy + low impact → fix if time
   - P4: Hard or low impact → create issues for hard items; fix trivial ones inline (doc comments, one-liners, undocumented edge cases)
   - **Write triage to `docs/audit-triage.md`** — fresh file with P1-P4 tables (include Status column). This survives context compaction.
   - **Carry forward prior-triage open items**: read the most recent archived triage (`docs/audit-triage-v*.md`, honoring any verification section that supersedes its row statuses), reconcile against PRs merged since, spot-grep ambiguous items against main, and append the still-open entries as CF-P2/CF-P3 tables. The live `docs/audit-triage.md` must be the single source of truth for everything open — no hunting through archives.

8. **Generate fix prompts**: For each P1, P2, and P3 finding, spawn fable agents to (P4 trivials get prompts too; hard P4s get issue descriptions):
   - Read the actual source file at the stated line numbers
   - Write a self-contained fix prompt with: exact file path, current code verbatim, replacement code, one-line "why"
   - Group related findings (e.g., stale doc references) into a single prompt
   - Save to `docs/audit-fix-prompts.md`

9. **Review fix prompts**: Spawn a second fable agent to verify each prompt against the actual source:
   - Does the "current code" match what's really in the file? (catches line drift)
   - Does the fix compile? (check types, imports, API existence)
   - Are there any missing edge cases?
   - Report: "VERIFIED" or "NEEDS FIX — [specific issue]"
   - This step catches 20-40% of prompt errors (wrong field names, nonexistent APIs, moved code) — measured NEEDS-FIX rates: P1 42%, P2 41%, P3 16%

10. **Execute fixes**: P1 first, then P2. Dispatch `opus` implementer agents for the fixes (implementation lane); keep verification/review agents on fable. Mark each item in triage as fixed.

11. **Disable audit mode**: `cqs audit-mode off`

## Category Scopes

| Category | Covers (merged from) |
|----------|---------------------|
| Code Quality | Dead code, duplication, complexity, coupling, cohesion, module boundaries, **convenience wrappers that hardcode defaults** |
| Documentation | Accuracy, completeness, staleness of docs and comments |
| API Design | Consistency, ergonomics, naming, type design |
| Error Handling | Result chains, context, recovery, swallowed errors |
| Observability | Logging coverage, tracing, debuggability |
| Test Coverage (adversarial) | Edge-case/sad-path gaps: malformed input, NaN/Inf embeddings, concurrent access, empty queries, huge inputs, error paths not tested |
| Test Coverage (happy path) | Missing tests for high-caller public functions, untested modules, integration test gaps, meaningful assertion quality |
| Scaling & Hardcoded Limits | Constants that should scale with model config, corpus size, or hardware. Magic numbers without rationale. |
| Robustness | unwrap/expect, edge cases (empty/huge/unicode/malformed), panic paths |
| Algorithm Correctness | Off-by-one, boundary conditions, logic errors |
| Extensibility | Adding features without surgery, hardcoded values |
| Platform Behavior | OS differences, path handling, WSL quirks |
| Security | Injection, path traversal, file permissions, secrets, access control |
| Data Safety | Corruption, validation, migrations, races, deadlocks, thread safety |
| Performance | O(n²), unnecessary iterations, batching, caching, I/O patterns |
| Resource Management | Memory usage, startup time, idle cost, OOM protection, leaks |

## Mandatory First Steps per Category

Run these cqs commands **before** manual exploration — they surface the highest-value data in a single call.

**Batch 1:**
- **Code Quality**: Run `cqs dead --json` + `cqs health --json` first. Also grep for convenience wrappers that hardcode defaults (e.g., `fn foo()` that calls `foo_with_dim(HARDCODED)`) — these mask incorrect wiring when the default changes.
- **Documentation**: Run `cqs health --json` for staleness counts.
- **API Design**: No mandatory command.
- **Error Handling**: No mandatory command — grep-driven.
- **Observability**: No mandatory command — grep for `tracing::` patterns.
- **Test Coverage (adversarial)**: Run `cqs health --json` first (includes untested hotspots). Check for **adversarial test gaps**: functions that accept user input, external data, or embeddings should have tests for malformed/adversarial inputs (empty, NaN, truncated, wrong-type, concurrent).
- **Robustness**: No mandatory command — grep for `.unwrap()`, `.expect(`, `panic!`.
- **Scaling & Hardcoded Limits**: No mandatory command — grep-driven (const definitions, `.clamp(` sites, capacity/Duration literals, dim literals).

**Batch 2:**
- **Algorithm Correctness**: Use `cqs explain <fn> --json` on algorithmic functions.
- **Extensibility**: Run `cqs health --json` for hotspot overview.
- **Platform Behavior**: No mandatory command.
- **Security**: No mandatory command.
- **Data Safety**: No mandatory command.
- **Performance**: Run `cqs health --json` first (identifies hotspots).
- **Resource Management**: No mandatory command.
- **Test Coverage (happy path)**: Run `cqs health --json` first (includes untested hotspots).

## Rules

- Collect ALL findings before fixing ANY
- One batch at a time (context limits)
- Clean up teams between batches
- Stop at diminishing returns during discovery
- Once triaged, complete the tier — don't suggest stopping mid-priority
- Cross-check findings against open GitHub issues — note overlaps as "existing #NNN"
- **Mark items in `docs/audit-triage.md` as fixed when done** — update the Status column (e.g., `✅ PR #N` or `✅ fixed`). This is the source of truth for what's been addressed.