Criteria Audit

---
name: criteria-audit
description: Validate EXECUTION_PLAN.md for verification metadata, manual reasons, and testability. Use when preparing Phase 1 or after editing EXECUTION_PLAN.md.
---

# Criteria Audit Skill

Audit EXECUTION_PLAN.md to ensure acceptance criteria are automation-ready and
use the verification metadata format.

## Arguments

- `$1` (optional) = directory containing `EXECUTION_PLAN.md`
  - If provided, read `$1/EXECUTION_PLAN.md`
  - If empty, read `EXECUTION_PLAN.md` from the current working directory

## Workflow Overview

Copy this checklist and track progress:

```
Criteria Audit Progress:
- [ ] Step 1: Read EXECUTION_PLAN.md
- [ ] Step 2: Parse phases, tasks, and acceptance criteria
- [ ] Step 3: Validate verification metadata
- [ ] Step 4: Report issues and summarize
```

## Step 1: Parse Acceptance Criteria

Resolve the plan path: if `$1` is provided, use `$1/EXECUTION_PLAN.md`; otherwise use `EXECUTION_PLAN.md` in the current working directory.

For each task, collect:
- Criterion text
- Type tag (e.g., `(TEST)`)
- `Verify:` line
- If manual: `Reason:` line

Also collect Pre-Phase Setup items and their `Verify:` lines.

## Step 2: Validation Rules

### Acceptance Criteria Rules

- Every criterion must include a type tag: `(TEST)`, `(CODE)`, `(LINT)`,
  `(TYPE)`, `(BUILD)`, `(SECURITY)`, `(BROWSER:DOM)`, `(MANUAL)`, `(MANUAL:DEFER)` etc.
- Every criterion must include a `Verify:` line unless it is `MANUAL` or `MANUAL:DEFER`.
- `MANUAL` and `MANUAL:DEFER` criteria must include a `Reason:` line.
- Flag ambiguous criteria (vague, subjective, or missing measurable details).

### Pre-Phase Setup Rules

- Each setup item must include a `Verify:` command.
- If missing, mark as human-required.

## Step 3: Check MANUAL Criteria for False Tags

Read `~/.claude/skills/auto-verify/PATTERNS.md` for the full pattern matching table
and MANUAL decision tree. If PATTERNS.md is not found, skip the false-MANUAL check and note the limitation in the report output (e.g., "PATTERNS.md not found — false-MANUAL detection skipped").

For each criterion tagged `(MANUAL)`, check if it contains keywords from the
Pattern Matching Table that indicate it CAN be automated (priorities 1-10).
If it matches any automatable pattern, it is a **false MANUAL tag**.

Only criteria matching the "Truly Manual Patterns" section (subjective UX/brand/tone
judgment) should remain as MANUAL.

## Step 3b: Check MANUAL Blocking Classification

For each `(MANUAL)` criterion (not `MANUAL:DEFER`):
- Check if it references subjective patterns WITH no downstream dependency
- If the criterion is purely cosmetic/tonal AND the next phase doesn't reference it:
  → Suggest retagging as `(MANUAL:DEFER)`
  → Reason: "No downstream dependency detected"

For each `(MANUAL:DEFER)` criterion:
- Verify it genuinely has no downstream dependency
- If a later task or phase references this criterion's output:
  → Flag as "Should be `MANUAL` (blocking) — downstream dependency exists"

## Step 4: Report

Provide a structured report:

```
CRITERIA AUDIT
==============

Tasks Checked: {N}
Criteria Checked: {N}
Issues Found: {N}

Missing Type Tags:
- Task 1.2.A: "{criterion}"

Missing Verify Lines:
- Task 1.3.B: "{criterion}"

Manual Missing Reason:
- Task 2.1.A: "{criterion}"

Pre-Phase Setup Missing Verify:
- Phase 1: "{setup item}"

False MANUAL Tags (should be automated):
- Task 1.2.A: (MANUAL) "{criterion}"
  → Suggest: (CODE) — Verify: `curl -sf {url} -o /dev/null`
  → Reason: Contains "endpoint"/"returns" — automatable via curl
- Task 2.1.B: (MANUAL) "{criterion}"
  → Suggest: (BROWSER:DOM) — Verify: route=`/page`, selector=`.class`
  → Reason: Contains "visible"/"displays" — automatable via browser

MANUAL Summary:
  Total MANUAL criteria: {N}
  Likely false tags: {N} (should be retagged to automated)
  Truly manual: {N} (subjective judgment)

DEFER Classification:
  Total MANUAL: {N} blocking, {M} deferrable
  Suggested retags: {list of MANUAL → MANUAL:DEFER or vice versa}

Status: PASS | WARN | FAIL
```

**FAIL** if any false MANUAL tags are found. **WARN** if MANUAL criteria exceed 10%
of total criteria. **PASS** otherwise.

## Error Handling

| Situation | Action |
|-----------|--------|
| EXECUTION_PLAN.md not found at the resolved path | Report "EXECUTION_PLAN.md not found" with the path checked and stop |
| PATTERNS.md not found (`~/.claude/skills/auto-verify/PATTERNS.md`) | Skip false-MANUAL detection, note limitation in report output |
| Plan file is empty or has no parseable phases/tasks | Report "No phases or tasks found" and mark audit as NOT APPLICABLE |
| A criterion has multiple conflicting type tags | Flag as ambiguous, list all tags found, and recommend the user pick one |
| Plan file exceeds 2000 lines | Process in chunks, warn user that partial analysis may miss cross-phase dependencies |

## Resolution Guidance

- If missing metadata is obvious, propose the exact type and `Verify:` line.
- For false MANUAL tags, propose the specific replacement type and verify command.
- If ambiguous, recommend asking the human to clarify.
- Do not edit EXECUTION_PLAN.md automatically unless explicitly requested.