Qe Snapshot Hygiene Auditor

---
name: wicked-garden-qe-snapshot-hygiene-auditor
description: |
  Snapshot-rot detector. Scans `__snapshots__/`, `*.snap`, `*.golden`,
  `cassettes/`, and `.syrupy` directories for four classes of rot:
  stale (>90d old AND still referenced), over-broad (full-DOM / full-JSON
  where a narrower assertion would do), rubber-stamped (baseline regenerated
  in the same commit that introduced the behavior change), and dead
  (file referenced by no active test). Outputs a ranked remediation list
  and defaults to a CONDITIONAL verdict with a top-N to re-review.

  Use when: snapshot audit, "our snap files are out of control", CI
  snapshot-update cleanup, reviewer fatigue triage, test-double rot check.
metadata:
  role: worker
---

# Snapshot Hygiene Auditor

Snapshot tests rot silently. An accepted-but-wrong baseline is worse than
a missing test because it pretends to be coverage. You audit the four
rot classes — stale, over-broad, rubber-stamped, dead — and produce a
ranked top-N list so a reviewer can spend 30 minutes, not 30 hours.

## Runtime
Script-backed steps use the `wicked-garden` launcher: `wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]`. It is on PATH after `npm i -g wicked-garden`; otherwise use `npx wicked-garden run …`; inside a wicked-crew run it is `"$WICKED_GARDEN_ROOT/scripts/wicked-garden"`.
If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is given next to it — never invent the script's output.
Relative paths in this skill are relative to the directory that contains this SKILL.md.

## 1. Inputs

- **Scenario file path** — frontmatter may declare:
  - `target_dirs:` list of directories to audit; default scan the whole
    repo for the patterns listed below.
  - `age_days:` stale threshold; default 90.
  - `over_broad_lines:` threshold flagging a single snapshot wider than
    N lines; default 200.
  - `rubber_stamp_window:` number of commits back to look for baseline-
    regenerated-same-commit patterns; default 1.
  - `top_n:` number of items in the remediation list; default 25.
- **`run_id`** — UUID of the current `runs` row; defines `EVIDENCE_DIR`.
- **Git working tree** — required; `git log -p` drives rubber-stamp
  detection. Refuse if the scenario's `target_dirs` are not inside a
  git repo.
- **`.wicked-qe/evidence/<run_id>/context.md`** — optional rules
  like "ignore cassettes/external/ — those are vendor fixtures".

## 2. Detector invocation

Run all four detectors; each writes its own JSON result to
`EVIDENCE_DIR/detectors/` for traceability.

### Discovery — what snapshot formats are in use

```bash
# The canonical snapshot-bearing patterns. Each match is a candidate for
# the four rot checks below.
#   - Jest / vitest: __snapshots__/*.snap
#   - Go / golden: *.golden, testdata/
#   - VCR / pytest-recording: cassettes/
#   - Syrupy (pytest): .syrupy/  or  __snapshots__/*.ambr
find "${TARGET_DIRS}" \
  \( -path '*/__snapshots__/*' -o -name '*.snap' -o -name '*.golden' \
     -o -path '*/cassettes/*' -o -path '*/.syrupy/*' -o -name '*.ambr' \) \
  -type f \
  > "${EVIDENCE_DIR}/snapshot-files.txt"
```

### Detector 1 — stale snapshots (>`age_days` AND still referenced)

```bash
# Age via git log of the snapshot file itself. A snapshot with no commit
# activity in N days that IS still referenced by a test is rot.
while read -r snap; do
  last_ts=$(git log -1 --format=%ct -- "${snap}" 2>/dev/null || echo 0)
  age_days=$(( ( $(date +%s) - last_ts ) / 86400 ))
  if [ "${age_days}" -gt "${AGE_DAYS}" ]; then
    # Is the snapshot still referenced by any *.test.* or *_test.* file?
    base=$(basename "${snap}" | sed -e 's/\.snap$//' -e 's/\.golden$//' -e 's/\.ambr$//')
    refs=$(git grep -l "${base}" -- '*.test.*' '*_test.*' 'test_*' | wc -l)
    [ "${refs}" -gt 0 ] && printf '%s\t%d\t%d\n' "${snap}" "${age_days}" "${refs}"
  fi
done < "${EVIDENCE_DIR}/snapshot-files.txt" \
  > "${EVIDENCE_DIR}/detectors/stale.tsv"
```

### Detector 2 — over-broad snapshots

```bash
# Long single-snapshot serializations are suspect: they assert on too many
# unrelated fields at once, so any change in any field forces a review.
while read -r snap; do
  lines=$(wc -l < "${snap}")
  if [ "${lines}" -gt "${OVER_BROAD_LINES}" ]; then
    printf '%s\t%d\n' "${snap}" "${lines}"
  fi
done < "${EVIDENCE_DIR}/snapshot-files.txt" \
  > "${EVIDENCE_DIR}/detectors/over-broad.tsv"
```

### Detector 3 — rubber-stamped snapshots

```bash
# Classic anti-pattern: the same commit that changes behavior also
# regenerates the baseline. That commit's diff will contain BOTH the
# source change AND the snapshot change.
node {WT_LIB}/snapshot/rubber-stamp.mjs \
  --window "${RUBBER_STAMP_WINDOW}" \
  --snapshot-files "${EVIDENCE_DIR}/snapshot-files.txt" \
  > "${EVIDENCE_DIR}/detectors/rubber-stamped.json"
```

`{WT_LIB}/snapshot/rubber-stamp.mjs` walks `git log -p` per snapshot, and
for each commit that touched the snap file, checks whether the same
commit ALSO touched a non-test source path in the same package. Matches
are surfaced with `commit_sha`, `source_paths_touched`, `snap_line_delta`.

### Detector 4 — dead snapshots

```bash
# A snapshot whose name/key is referenced by no active test.
while read -r snap; do
  base=$(basename "${snap}" | sed -e 's/\.snap$//' -e 's/\.golden$//' -e 's/\.ambr$//')
  refs=$(git grep -l "${base}" -- '*.test.*' '*_test.*' 'test_*' 'spec/**' | wc -l)
  [ "${refs}" -eq 0 ] && printf '%s\n' "${snap}"
done < "${EVIDENCE_DIR}/snapshot-files.txt" \
  > "${EVIDENCE_DIR}/detectors/dead.txt"
```

## 3. Scoring + ranking

Each flagged snapshot gets a score from the detectors; the top-N is
sorted descending.

```
score = (stale ? 1 : 0) * 2
      + (over_broad_lines > threshold ? 1 : 0) * 1
      + (rubber_stamped ? 1 : 0) * 3      # highest weight — false confidence
      + (dead ? 1 : 0) * 1
```

Ties break by `lines` descending (larger files first — more reviewer
leverage).

## 4. Evidence output

Under `.wicked-qe/evidence/<run_id>/`:

| File                              | manifest `kind` | Required |
|-----------------------------------|-----------------|----------|
| `snapshot-files.txt`              | `log`           | Yes      |
| `detectors/stale.tsv`             | `log`           | Yes      |
| `detectors/over-broad.tsv`        | `log`           | Yes      |
| `detectors/rubber-stamped.json`   | `log`           | Yes      |
| `detectors/dead.txt`              | `log`           | Yes      |
| `snapshot-audit.md`               | `log`           | Yes      |
| `snapshot-top-n.csv`              | `log`           | Yes      |

`snapshot-audit.md` has one section per detector with summary counts,
then a `## Remediation` section listing the top-N with per-item: path,
score, which detectors fired, age, lines, rubber-stamp commit sha (if
any), suggested action (narrow / delete / regenerate-with-review).

`snapshot-top-n.csv` columns:
`path,score,stale,over_broad,rubber_stamped,dead,age_days,lines,commit_sha,action`.

## 5. DomainStore write

```js
// Default to CONDITIONAL — this audit is advisory unless the scenario
// escalates the threshold. A FAIL verdict is issued when rubber-stamped
// rot crosses a hard limit (see rules below).
const anyRubberStamped = rubberStamped.length > 0;
const severeRot = rubberStamped.length > 25 || dead.length > 100;
const verdict = severeRot ? "FAIL" : "CONDITIONAL";

store.create("verdicts", {
  run_id: RUN_ID,
  verdict,
  reviewer: "wicked-garden-qe-snapshot-hygiene-auditor",
  reason: `stale=${stale.length} over_broad=${overBroad.length} rubber_stamped=${rubberStamped.length} dead=${dead.length}; top_n=${topN.length}. See snapshot-audit.md for the ranked remediation list.`,
  evidence_path: `evidence/${RUN_ID}/`,
  // Inside a governed wicked-crew run, name it, so crew attributes this verdict by stamp.
  ...(process.env.WICKED_RUN_ID ? { crew_run_id: process.env.WICKED_RUN_ID } : {}),
});

// One open task per top-N item so they surface in the reviewer's queue.
for (const item of topN) {
  store.create("tasks", {
    project_id: PROJECT_ID,
    title: `Snapshot rot: ${item.path} (score ${item.score})`,
    status: "open",
    assignee_skill: "snapshot-hygiene-auditor:remediation",
    body: JSON.stringify({
      path: item.path, score: item.score,
      detectors: item.detectors,         // [stale, over_broad, rubber_stamped, dead]
      age_days: item.ageDays, lines: item.lines,
      commit_sha: item.commitSha || null,
      action: item.action,               // narrow | delete | regenerate-with-review
    }),
  });
}
```

## 6. Failure modes

| code                          | meaning                                             | class  |
|-------------------------------|-----------------------------------------------------|--------|
| `ERR_NOT_A_GIT_REPO`          | `target_dirs` are outside a git working tree        | user   |
| `ERR_NO_SNAPSHOTS_FOUND`      | zero snapshot files detected — scenario is noop     | user   |
| `ERR_TARGET_DIR_MISSING`      | declared `target_dirs` path doesn't exist           | user   |
| `ERR_GIT_LOG_FAILED`          | git log subprocess failed (shallow clone?)          | system |
| `ERR_EVIDENCE_DIR_MISSING`    | evidence dir not pre-created                        | system |

On `ERR_NO_SNAPSHOTS_FOUND`: the scenario is a noop — don't fabricate
findings. Return a PASS with `reason: "no snapshots found"` and
`found=0`.

## 7. Non-negotiable rules

- **Default ledger verdict is CONDITIONAL; the output line is then
  `VERDICT: FAIL` with the top-N to re-review as conditions.** This is a
  reviewer tool, not a gate — but there is no conditional pass on the output
  line: `VERDICT: PASS` only for a clean tree (or the `ERR_NO_SNAPSHOTS_FOUND`
  noop). The ledger's FAIL is reserved for severe rubber-stamp counts or
  catastrophic dead-file counts.
- **Rubber-stamp detection requires git history.** Shallow clones with
  depth < `rubber_stamp_window + 5` trigger `ERR_GIT_LOG_FAILED`; do
  not guess.
- **Never delete snapshots.** Output a task with `action: delete`; the
  human executes it.
- **Respect vendor-fixture ignores.** `cassettes/external/` or any path
  added to context.md's ignore list is excluded from all four detectors.
- **Top-N order is stable per run.** Given the same inputs, the CSV
  ordering must be identical — auditors diff across runs.

## 8. Output

The final line is exactly `VERDICT: PASS` or `VERDICT: FAIL` — one plain-text
line, nothing after it, never quoting another VERDICT line.

```
## Snapshot audit: {scenario.name}
target_dirs: {target_dirs}
thresholds: age_days={ageDays} over_broad_lines={obLines} rubber_stamp_window={rsWindow}

found: {total} snapshot files

detector counts:
  stale         : {stale}
  over-broad    : {overBroad}
  rubber-stamped: {rubberStamped}     <-- highest weight
  dead          : {dead}

top-5 (full list in snapshot-top-n.csv):
  score=7 src/ui/__snapshots__/Header.snap    (stale+over-broad+rubber-stamped)  action=regenerate-with-review
  score=5 src/pricing/testdata/tax.golden     (stale+rubber-stamped)             action=regenerate-with-review
  score=3 tests/cassettes/legacy/old.yaml     (dead)                              action=delete
  ...

REVIEWER: wicked-garden-qe-snapshot-hygiene-auditor
RUN_ID: {RUN_ID}
VERDICT: PASS|FAIL
```

## Helper resolution (`{WT_LIB}`)

`{WT_LIB}` is the plugin's own qe helper directory — the helper modules ship
in-catalog (`scripts/qe/lib/`, ported from the retired wicked-testing package <!-- historical -->
in Phase 6c). Resolve it (cross-platform):

```bash
WT_LIB="$(wicked-garden path scripts/qe/lib)"
```