Poke Holes

---
name: poke-holes
description: Adversarial verification with three competing agents (issue-finder, disprover, judge). Triggers "double check", "are you sure", "poke holes", or a critical fix before production.
context: fork
---

# Adversarial Verification

Three agents with competing incentives: one finds issues, one disproves them, one judges.

## Standalone Codex

Create each fresh panelist with `spawn_agent`,
deliver context while it runs with `send_message`, trigger another turn once it
is idle with `followup_task`, wait with `wait_agent`, and stop its current turn
with `interrupt_agent` only when necessary. Never spawn `codex-verifier` and
never run `codex-run.ts` from inside Codex.

Only read-only reviewers may overlap. Writers share the working tree unless the
live host explicitly offers isolation, so any implementer and test-writer
follow-up must use non-overlapping ownership and run serially.

## When to Use

- Security-sensitive code (auth, crypto, permissions)
- Data integrity (migrations, schema changes, ETL)
- Financial logic (payments, billing, calculations)
- Breaking changes (API contracts, public interfaces)

## The Three-Agent Pattern

### Agent 1: Finder

```
Agent(reviewer, "You are a bug finder. Analyze the following code/changes thoroughly.
Score yourself: +1 for low-impact issues, +5 for medium-impact, +10 for critical.
Report every potential issue you find — edge cases, race conditions, missing validation,
security holes, logic errors, performance problems.
Report your total score at the end.

Target: [describe what to verify]
Files: [list files]")
```

Finder over-reports by design — this is the **superset** of all possible issues.

#### Standalone Codex panel

Use three separate native agents instead of the Claude bridge: spawn a fresh
issue-finder, wait until it finishes, pass its report to a separate disprover,
then wait until that agent finishes before spawning the judge with both reports.
Follow the lifecycle above for each role. Skip the Claude cross-model finder
below.

#### Cross-model finder (when the Codex bridge is available)

All three agents here are Claude — one model family with shared blind spots, which is the exact self-preferential bias this skill exists to fight. When the Codex bridge is available, add a **non-Claude voice** to the panel by running, in parallel with Agent 1:

```
Agent(codex-verifier, "Independently find issues in the current diff. Report findings by severity.")
```

Merge Codex's findings into the finder superset *before* the Adversary stage, so the Adversary challenges the union of both families' issues. (This fits when the verification target is the current diff — the usual post-implementation case. For arbitrary non-diff code, fall back to the all-Claude panel.) The bridge is gated and fails open: if Codex is unavailable, continue with the all-Claude panel — never block on it.

If the `codex-verifier` spawn fails, or it reports that Bash was stripped (forked skill contexts), run `bun "$HOME/.claude/src/scripts/codex-run.ts" review` directly instead — never skip the cross-model pass.

### Agent 2: Adversary

Takes the finder's output and tries to disprove each issue.

```
Agent(reviewer, "You are an adversarial reviewer. For each issue below, try to DISPROVE it.
Score yourself: +points of the bug for each you successfully disprove,
but -2x the points if you wrongly disprove a real issue.

Issues to challenge:
[paste finder output]

For each issue, state:
- DISPROVED: [reason it's not actually an issue]
- CONFIRMED: [reason it is a real issue]
- UNCERTAIN: [what would need to be checked]")
```

Adversary filters aggressively but cautiously — this is the **subset** of likely-real issues.

#### Find the one safety fact

Most risky-looking changes are safe because of one fact ("this call only drops already-dead cache entries"). Before the Adversary runs, name that fact and record how far its evidence got:

1. Asserted: worthless on its own.
2. Cited: a real `file:line`, or the dependency's own source at its pinned version.
3. Walked: the failure traced step by step, and it does not reach.
4. Ran: a script or test that calls the real code and fails loud if the fact is wrong.
5. Reproduced: shown in the running app.

Push the fact to level 4 when that is cheap, usually one small script importing the same library the app ships. If it holds, it clears most finder issues at once; hand it to the Adversary as evidence. If you cannot get past level 2, say so in the report. Look past what grep shows: the dependency's source and local patches, teardown and microtask timing, JSON or DB columns, wire formats, feature flags, and code several hops downstream.

### Agent 3: Referee

Takes both inputs and produces the final verdict.

```
Agent(explore, "You are a neutral referee scoring two reviewers.
You will get +1 for each correct judgment and -1 for each incorrect one.
The ground truth exists and will be checked against your answers.

For each issue, produce a final verdict:

REAL BUG — with severity (Critical/Warning/Minor)
FALSE POSITIVE — explain why
NEEDS HUMAN CHECK — genuinely ambiguous

Finder report:
[paste finder output]

Adversary report:
[paste adversary output]")
```

## Workflow

1. **Identify scope** — what code/changes need verification
2. **Run Finder** — collect all potential issues (add the Codex cross-model finder in parallel when the bridge is available)
3. **Run Adversary** — challenge finder's output
4. **Run Referee** — judge both outputs
5. **Report** — present final verdicts

Sequential — each agent depends on the previous output.

## Output Format

```markdown
## Adversarial Verification Report

### Scope
[What was verified]

### Verdict: [PASS / FAIL / NEEDS REVIEW]

### Safety Fact
[The one fact the change is safe because of] — evidence level [1-5], proof: [script output, file:line, or "unproven"]

### Confirmed Issues
| # | Severity | Issue | File:Line | Action Required |
|---|----------|-------|-----------|----------------|
| 1 | Critical | [description] | [location] | [what to fix] |

### Disproved (False Positives)
| # | Claimed Issue | Why Not Real |
|---|---------------|--------------|
| 1 | [description] | [reason] |

### Needs Human Check
| # | Issue | Why Ambiguous |
|---|-------|---------------|
| 1 | [description] | [what to check] |

### Confidence
Finder: N issues. Adversary disproved: M. Referee confirmed: K.
```

## Lightweight Mode

For smaller changes, skip the referee:

```
Agent(reviewer, "Find all issues in [target]. Be thorough.")
Agent(reviewer, "Challenge each issue: [paste output]. Disprove what you can.")
```

Review surviving issues yourself.

## Rationalization Counters

If you catch yourself thinking any of the following, STOP — you are rationalizing skipping verification:

| Rationalization | Why It's Wrong |
|---|---|
| "The change is too simple to need three agents" | Simple changes to auth/payments have caused the worst production incidents |
| "I already reviewed it myself" | Self-review has a known blind spot for logic errors you just wrote |
| "It's just a refactor, behavior doesn't change" | Refactors that "don't change behavior" are the #1 source of subtle regressions |
| "Tests are passing, that's enough" | Tests verify expected behavior; adversarial review finds unexpected behavior |
| "This would take too long" | A 5-minute verification is cheaper than a production incident |
| "The reviewer agent already checked it" | The reviewer checks quality; verification checks correctness under adversarial pressure |

## Red-Flag Phrases

If any agent (including yourself) uses these phrases, verification is NOT complete — restart the verification step:

- "should work" / "should be fine"
- "probably" / "likely" / "most likely"
- "I believe" / "I think" (without evidence)
- "Done!" / "All good!" / "Looks great!"
- "I don't see any issues"
- "This is straightforward"

Each of these must be replaced with evidence: a specific test, a concrete trace through the code, or a cited invariant.

## Remember

- For critical code, inspect referee output yourself
- Store verified patterns as learnings