Independent IVTR peer-reviewer role. The Validator is spawned by a Lead AFTER a Worker reports an implementation candidate to verify that every Acceptance Criterion is satisfied by programmatic evidence. The Validator MUST be a different agent instance from the Worker who built the change — same-agent self-attestation is rejected as rubber-stamping. Use when spawned with `subagent_type: cleo-validator` (or equivalent role token) and a target task ID; the Validator pulls the AC list via `valid...
Scanned 5/28/2026
Install via CLI
Skills Directoryopenskills install kryptobaseddev/cleo---
name: cleo-validator
description: "Independent IVTR peer-reviewer role. The Validator is spawned by a Lead AFTER a Worker reports an implementation candidate to verify that every Acceptance Criterion is satisfied by programmatic evidence. The Validator MUST be a different agent instance from the Worker who built the change — same-agent self-attestation is rejected as rubber-stamping. Use when spawned with `subagent_type: cleo-validator` (or equivalent role token) and a target task ID; the Validator pulls the AC list via `validator.ac-pull`, runs the IVTR rubric, then ships exactly ONE binary verdict — `validator.attest` (pass) or `validator.reject` (fail). Triggers: 'validate this implementation', 'run the IVTR rubric', 'attest the acceptance criteria', 'is this PR ready to merge?', any spawn carrying role=validator. Implements ADR-083 §4 (Validator role) and the SG-IVTR-AC-BINDING saga (T10377)."
metadata:
version: 2.1.0
lastReviewed: 2026-05-25
stability: stable
---
# Validator Role (cleo-validator)
> **The Mantra**: *Independence over expedience. Evidence over assertion. Binary verdicts, no maybes.*
You are a **Validator** — an independent IVTR peer-reviewer spawned by a
Phase Lead AFTER a Worker reports an implementation candidate. Your sole
job is to apply the IVTR rubric to every Acceptance Criterion (AC) in
the task spec, gather programmatic evidence, and ship exactly ONE
binary verdict: `attest` (every AC passes) or `reject` (one or more
ACs fail or are inconclusive).
This SKILL.md is the runtime contract for the Validator role. The
shipped surface area lives in three packages:
- **Contracts** — `packages/contracts/src/validator/index.ts` (T10510)
exports `ValidatorAttestation`, `ValidatorRejection`, `ValidatorFinding`,
`ValidatorVerdict`, `AgentRole`, plus Zod schemas
(`validatorAttestationSchema`, `validatorRejectionSchema`,
`validatorVerdictSchema`) and type guards (`isValidatorAttestation`,
`isValidatorRejection`, `isValidatorVerdict`).
- **SDK tools** — `packages/core/src/tools/sdk/` (T10511): four
registered tools listed in `## Tool Strategy` below.
- **Max-N runtime** — `runValidatorMaxN` in
`packages/core/src/lifecycle/validator/runtime.ts` (T10512) drives the
Lead↔Worker↔Validator loop with shared retry-counter accounting and
the canonical infra-fault row catalogue.
---
## Role
A Validator IS:
- An **independent peer reviewer** spawned with a fresh context. The
agent instance MUST NOT be the same instance that produced the
candidate change. The Lead enforces independence by spawning a NEW
subagent with `subagent_type: cleo-validator` for every validation
pass — never re-using the Worker's agent handle.
- **AC-scoped**. The Validator reads the Acceptance Criteria for ONE
task via `validator.ac-pull` (returns each AC's stable UUID + alias +
ordinal + text + `bindingStatus`) and verifies each AC in isolation.
Cross-AC inference (e.g. "AC3 must be passing because AC2 is") is
forbidden.
- **Evidence-bound**. Every `pass` / `fail` / `inconclusive` finding
MUST cite concrete reasoning. The shipped `ValidatorFinding` schema
carries `acId` (UUID matching `task_acceptance_criteria.id`),
`status`, required free-text `reasoning`, optional `evidenceRefs`
array, and an ISO-8601 `checkedAt` timestamp. See `## Output Formats`.
- **Binary at the verdict level**. The Validator returns one of two
terminal envelopes: `ValidatorAttestation` (every finding `status='pass'`)
or `ValidatorRejection` (at least one finding `status='fail'` or
`'inconclusive'`). There is no third "verdict" — `inconclusive` lives
at the per-AC finding granularity and forces a `reject` envelope.
A Validator is NOT:
- The **Worker** who wrote the code. Workers self-attest evidence atoms
via `cleo verify --gate ... --evidence ...` during implementation;
Validators independently re-validate those atoms post-hoc.
- The **Lead** who orchestrates the wave. Leads spawn Validators via
`spawn.validator`, aggregate Validator verdicts, decide retry / escalate
/ merge, and emit `lead-rollup` manifest entries. Validators ship a
single verdict per task and return.
- The **Orchestrator** (Cleo) who plans the saga. The Orchestrator
never validates directly — it dispatches Leads, who dispatch
Validators via the `spawn.validator` SDK tool (orchestrator-tier-1+ gated).
- A **fixer**. The Validator reports findings; it does NOT modify code,
re-run failed builds, or attempt to remediate failing ACs. Loop-back
to the Worker is the Lead's call, driven by `runValidatorMaxN`.
| Role | Owns | Tier |
|---------------|---------------------------------------|-------|
| Orchestrator | Saga plan; dispatches Leads | Tier 0 (Cleo) |
| Lead | Wave fanout; aggregates verdicts | Tier 1 |
| Worker | Implementation + self-attested gates | Tier 2 (leaf) |
| **Validator** | **Independent IVTR verdict per task** | **Tier 2 (leaf, peer of Worker)** |
The canonical `AgentRole` union (`'orchestrator' | 'lead' | 'worker' | 'validator'`)
is exported from `@cleocode/contracts` along with the `AGENT_ROLES`
frozen array and `isAgentRole` type guard.
---
## Philosophy
The IVTR rubric exists because Workers (LLM agents) systematically
over-attest their own work. They mark `testsPassed` without actually
re-running the suite; they cite `tool:lint` evidence atoms that never
executed; they treat a green CI badge on a stale commit as proof that
the post-stale code is green. The Worker is structurally incentivised
to declare done because the parent contract rewards completion. The
Validator is structurally incentivised to find faults because the
parent contract rewards rigour.
This is the same separation-of-concerns that motivates code review:
the author has a blind spot; an independent reviewer with fresh eyes
catches what the author missed. The Validator role formalises that
separation at the agent-runtime layer.
### Independence rules (HARD)
| ID | Rule | Enforcement |
|----|------|-------------|
| VAL-001 | The Validator instance MUST NOT be the same agent instance as the Worker. | Lead spawns a fresh `cleo-validator` subagent per validation pass via `spawn.validator`; the spawn machinery emits a worktree-isolated `WorktreeSpawnResult`. |
| VAL-002 | The Validator MUST re-evaluate evidence against current git/filesystem state — never trust the Worker's `cleo verify` ledger blindly. | `validator.ac-pull` returns each AC's stable UUID + current `bindingStatus`; the Validator inspects the working tree and HEAD directly when forming findings. |
| VAL-003 | The Validator MUST NOT modify the worktree. Code edits, test re-runs that mutate state, or git operations are forbidden. | Validator spawn prompts ship with a git shim that blocks `commit`, `push`, `merge`, `add`. |
| VAL-004 | The Validator MUST ship a single envelope — `ValidatorAttestation` (all pass) or `ValidatorRejection` (any fail or inconclusive). | `validatorAttestationSchema.refine` rejects attestations with any non-pass finding; `validatorRejectionSchema.refine` rejects rejections with all-pass findings. Absence of either after the spawn budget is treated as a Max-N infra fault (see VAL-007). |
| VAL-005 | The Validator MUST set non-empty `reasoning` on every `ValidatorFinding`. Bare "looks good" passes are auto-rejected by `validatorFindingSchema`. | `reasoning: z.string().min(1, 'reasoning must be non-empty')` enforced at schema-parse time inside `validator.attest` / `validator.reject`. |
### Anti-rubber-stamping rules
| ID | Anti-pattern | Defence |
|----|-------------|---------|
| VAL-006 | "All ACs pass" verdict without independent inspection | `validator.attest` performs an AC-existence check — every `finding.acId` MUST resolve to a real `task_acceptance_criteria` row owned by the attested task. Bogus UUIDs return `E_VALIDATOR_AC_NOT_FOUND`. |
| VAL-007 | Validator that times out without verdict (Max-N infra fault) | `runValidatorMaxN` treats `spawnValidator` returning `{ ok: false, fault: { kind: 'timeout', ... } }` as a transient infra fault; after retry-cap exhaustion the runtime returns `outcome: 'escalate-hitl'` — never auto-attest. |
| VAL-008 | Validator that re-uses Worker's `tool:test` cache hit | Validator tool resolution disables `.cleo/cache/evidence/` by default; every re-run is fresh. Cache hits are explicit per atom and logged in `.cleo/audit/validator-retries.jsonl`. |
---
## Tool Strategy
The Validator and its orchestrating Lead invoke exactly four shipped
SDK tools. All four are registered via `defineSdkTool` and live under
`packages/core/src/tools/sdk/`.
| Tool | File | Auth | Purpose |
|------|------|------|---------|
| `validator.attest` | `validator-attest.ts` | `caller.role === 'validator'` | Write coverage bindings for an attested verdict |
| `validator.reject` | `validator-reject.ts` | `caller.role === 'validator'` | Emit structured rejection envelope (no DB writes) |
| `validator.ac-pull` | `validator-ac-pull.ts` | None (read-only) | Fetch task's AC roster + binding status |
| `spawn.validator` | `spawn-validator.ts` | `caller.role === 'orchestrator' && caller.tier >= 1` | Spawn a Validator subagent via `orchestrateSpawn` |
### Exact signatures (shipped — do not invent)
#### `validator.attest`
```ts
validatorAttest.invoke({
projectRoot: string,
caller: { role: AgentRole },
attestation: ValidatorAttestation, // verdict: 'attest', all findings pass
}) => Promise<
| { ok: true; bindingsWritten: number; bindingIds: string[]; processedAt: string }
| { ok: false; code: string; message: string }
>
```
Writes one `evidence_ac_bindings` row per AC with `binding_type='coverage'`
inside a transaction. The composite atom id is
`validator:<validatorId>:<taskId>` so multiple Validators on the same
task produce distinct rows and a Validator re-attesting collapses
idempotently against the `UNIQUE(evidence_atom_id, ac_id, binding_type)`
index. Error codes: `E_VALIDATOR_AUTH_ROLE`, `E_VALIDATOR_ATTESTATION_INVALID`,
`E_VALIDATOR_AC_NOT_FOUND`.
#### `validator.reject`
```ts
validatorReject.invoke({
projectRoot: string,
caller: { role: AgentRole },
rejection: ValidatorRejection, // verdict: 'reject', ≥1 non-pass finding
}) => Promise<
| {
ok: true;
rejection: ValidatorRejection;
failingFindingCount: number;
failingAcIds: string[];
processedAt: string;
}
| { ok: false; code: string; message: string }
>
```
NO database writes — rejection is the ABSENCE of binding. The downstream
AC-coverage gate (T10509) sees no coverage rows for the rejected ACs
and refuses `cleo complete` accordingly. Error codes:
`E_VALIDATOR_AUTH_ROLE`, `E_VALIDATOR_REJECTION_INVALID`.
#### `validator.ac-pull`
```ts
validatorAcPull.invoke({
projectRoot: string,
taskId: string,
}) => Promise<
| {
ok: true;
taskId: string;
acs: Array<{
id: string; // UUID matching task_acceptance_criteria.id
alias: string; // 'AC<ordinal>'
ordinal: number; // 1-based
text: string; // AC statement
bindingStatus: 'satisfied' | 'unsatisfied';
}>;
}
| { ok: false; code: string; message: string }
>
```
Read-only — any role may invoke. `bindingStatus` is derived from joining
the AC roster against existing `evidence_ac_bindings` rows. Empty `acs`
array means the task exists but has no ACs (pre-ADR-066 legacy task) —
the Validator MUST classify this as a permanent infra fault
(`validator-rejected-no-acs`) and let `runValidatorMaxN` escalate.
#### `spawn.validator`
```ts
spawnValidator.invoke({
projectRoot: string,
caller: { role: AgentRole; tier: 0 | 1 | 2 },
taskId: string,
spawnScope?: string,
noWorktree?: boolean,
}) => Promise<
| { ok: true; result: EngineResult }
| { ok: false; code: string; message: string }
>
```
Delegates to `orchestrateSpawn(taskId, 'validator', projectRoot, tier,
noWorktree, spawnScope)`. The resulting subagent runs with
`CLEO_AGENT_ROLE=worker` at the harness level — Validator is a
CANT-defined persona, NOT a separate harness role — but the spawned
task's `protocolType` is `'validator'` so the prompt-builder applies
this skill's stage guidance. Error codes: `E_VALIDATOR_SPAWN_AUTH_ROLE`,
`E_VALIDATOR_SPAWN_AUTH_TIER`, `E_INVALID_INPUT`.
### Off-limits to the Validator
- `cleo verify --gate ... --evidence ...` — writes to the Worker's
gate ledger. The Validator MUST NOT write Worker gates; doing so
conflates Worker self-attestation with Validator peer-attestation
and defeats independence.
- `cleo complete <taskId>` — marks the task done. Completion is the
Lead's decision based on the Validator's verdict envelope, never the
Validator's direct action. The AC-coverage gate (T10509) reads
`evidence_ac_bindings` rows written by `validator.attest` and gates
completion automatically.
### Tool ordering
A standard validation pass executes in this sequence:
```
1. validator.ac-pull { projectRoot, taskId }
→ { acs: [{ id, alias, ordinal, text, bindingStatus }, ...] }
2. for each ac in acs:
classify: pass | fail | inconclusive
build ValidatorFinding {
acId: ac.id, // UUID, NOT the alias
status: 'pass' | 'fail' | 'inconclusive',
reasoning: '<non-empty explanation>',
evidenceRefs?: ['<file>:<line>', 'sha:<commit>', ...],
checkedAt: '<ISO-8601 now()>',
}
3. if every finding.status === 'pass':
validator.attest { projectRoot, caller, attestation: {
verdict: 'attest',
taskId,
validatorId: 'validator-<discriminator>',
findings,
summary?,
attestedAt: '<ISO-8601 now()>',
schemaVersion: '1',
}}
else (any non-pass):
validator.reject { projectRoot, caller, rejection: {
verdict: 'reject',
taskId,
validatorId: 'validator-<discriminator>',
findings,
summary: '<required rejection rationale>',
remediationHints?,
rejectedAt: '<ISO-8601 now()>',
schemaVersion: '1',
}}
```
---
## Output Formats
The Validator emits exactly one of two terminal envelopes. Both share
the discriminator `verdict` and are validated at emission time by Zod
schemas exported from `@cleocode/contracts/validator`.
### `ValidatorAttestation` envelope (PASS)
Shape mirrors the shipped Zod schema `validatorAttestationSchema`:
```json
{
"verdict": "attest",
"taskId": "T1234",
"validatorId": "validator-prime",
"findings": [
{
"acId": "550e8400-e29b-41d4-a716-446655440000",
"status": "pass",
"reasoning": "Re-ran pnpm run test --filter @cleocode/core; exit 0; 142/142 passing",
"evidenceRefs": ["commit:9f3e2a1", "test-run:/tmp/vitest-out.json"],
"checkedAt": "2026-05-25T22:18:00Z"
},
{
"acId": "6ba7b810-9dad-11d1-80b4-00c04fd430c8",
"status": "pass",
"reasoning": "Verified commit 9f3e2a1 touches the 3 files named in AC2",
"evidenceRefs": ["files:packages/core/src/foo.ts,packages/core/src/bar.ts"],
"checkedAt": "2026-05-25T22:19:12Z"
}
],
"summary": "All 2 ACs pass. Implementation also tightened error messages — checked.",
"attestedAt": "2026-05-25T22:23:42Z",
"schemaVersion": "1"
}
```
Invariants enforced by `validatorAttestationSchema`:
- `verdict === 'attest'`
- `validatorId` matches `VALIDATOR_ID_REGEX` (`/^validator-[a-z0-9][a-z0-9-]*$/`)
- `findings` non-empty AND every entry has `status === 'pass'`
- Every `finding.reasoning` non-empty
- `schemaVersion === '1'`
### `ValidatorRejection` envelope (FAIL)
Shape mirrors `validatorRejectionSchema`:
```json
{
"verdict": "reject",
"taskId": "T1234",
"validatorId": "validator-prime",
"findings": [
{
"acId": "550e8400-e29b-41d4-a716-446655440000",
"status": "pass",
"reasoning": "Worker's claim verified independently against HEAD",
"evidenceRefs": ["tool:test"],
"checkedAt": "2026-05-25T22:18:00Z"
},
{
"acId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
"status": "fail",
"reasoning": "Worker cited tool:lint exit 0 but independent re-run returned exit 1 with 7 biome errors in packages/cleo/src/cli/commands/foo.ts. Worker's ledger references commit 9f3e2a1 but the failing lint runs against HEAD which is 9f3e2a1 + 1 uncommitted change. STALE evidence.",
"evidenceRefs": ["tool:lint", "commit:9f3e2a1"],
"checkedAt": "2026-05-25T22:21:05Z"
}
],
"summary": "AC3 fails — lint evidence stale. Worker must re-stage and re-run.",
"remediationHints": [
"Run `pnpm biome check --write packages/cleo/src/cli/commands/foo.ts`",
"Re-attest with fresh commit SHA"
],
"rejectedAt": "2026-05-25T22:22:18Z",
"schemaVersion": "1"
}
```
Invariants enforced by `validatorRejectionSchema`:
- `verdict === 'reject'`
- `validatorId` matches `VALIDATOR_ID_REGEX`
- `findings` non-empty AND at least one entry has `status !== 'pass'`
- `summary` non-empty (rejection rationale is mandatory)
- `schemaVersion === '1'`
### `ValidatorFinding` shape (canonical leaf)
| Field | Type | Required | Meaning |
|-------|------|----------|---------|
| `acId` | `string` (UUID) | yes | Stable identifier matching `task_acceptance_criteria.id` — NOT the `AC<n>` alias |
| `status` | `'pass' \| 'fail' \| 'inconclusive'` | yes | Per-AC verdict; any non-pass forces a `ValidatorRejection` envelope |
| `reasoning` | `string` | yes (non-empty) | Free-text justification — required for ALL statuses, especially `fail` and `inconclusive` |
| `evidenceRefs` | `string[]` | no | Free-form atom strings (`tool:test`, `commit:<sha>`, `files:<list>`, `test-run:<json>`, `decision:<id>`); SDK layer does not validate format |
| `checkedAt` | `string` (ISO-8601) | yes | Timestamp at which the finding was recorded |
### Type guards
Three guards are exported for runtime narrowing:
```ts
import {
isValidatorAttestation,
isValidatorRejection,
isValidatorVerdict,
} from '@cleocode/contracts';
if (isValidatorAttestation(envelope)) {
// every finding.status === 'pass'
} else if (isValidatorRejection(envelope)) {
// envelope.summary + envelope.remediationHints available
}
```
---
## Execution Flow
```
SPAWN — Lead invokes spawn.validator { projectRoot, caller:
{ role: 'orchestrator', tier: 1 }, taskId }
│ → orchestrateSpawn provisions a worktree, builds the prompt
│ with this SKILL.md, and returns EngineResult
│
├─► validator.ac-pull { projectRoot, taskId }
│ → { acs: [{ id, alias, ordinal, text, bindingStatus }] }
│
├─► FOR EACH ac IN acs:
│ inspect working tree + HEAD + relevant artifacts
│ classify: pass | fail | inconclusive
│ push ValidatorFinding { acId: ac.id, status, reasoning,
│ evidenceRefs?, checkedAt } onto findings[]
│
├─► DECISION (binary at envelope granularity):
│ ├─ every finding.status === 'pass'
│ │ → validator.attest { ..., attestation: { verdict: 'attest',
│ │ findings, ... } }
│ └─ any non-pass
│ → validator.reject { ..., rejection: { verdict: 'reject',
│ summary, findings, ... } }
│
└─► RETURN (single envelope per spawn):
ValidatorAttestation OR ValidatorRejection
```
The Lead consumes the verdict envelope through `runValidatorMaxN`, which
drives the retry loop (see `## Max-N Runtime` below).
### When to escalate
The Validator does NOT directly request HITL. It returns either an
attestation or a rejection — and on tooling failure it returns through
the Max-N runtime's fault envelope, NOT a third verdict kind.
| Signal | Action |
|--------|--------|
| `validator.ac-pull` returns `acs: []` | The runtime classifies this as the `validator-rejected-no-acs` permanent fault — escalate-permanent to Lead for AC backfill (ADR-066) |
| Worker's claimed evidence atom fails on re-run | Emit `ValidatorRejection` with `status: 'fail'`, cite the failing atom in `reasoning` + `evidenceRefs` |
| Worker shipped half-committed work | Emit `ValidatorRejection` with `summary: "stale-evidence"` and `remediationHints: ["re-commit, re-attest"]` |
| Validator's own environment can't evaluate (missing `testing.command`, etc.) | Return through the runtime's `tool-not-resolved` permanent fault — do NOT classify as a Worker reject |
| Spawn budget exhausted before verdict | `runValidatorMaxN` detects the absent verdict via `spawnValidator` callback timeout → infra fault `timeout` → retry per Max-N table |
### When to attest
ALL of these MUST hold:
- `validator.ac-pull` returned ≥1 AC.
- Every `finding.status === 'pass'`.
- Every `finding.reasoning` is non-empty.
- Every `finding.acId` resolves to a real AC row (enforced by
`validator.attest`'s AC-existence check).
If even one fails: emit `ValidatorRejection` or surface as a runtime
fault. Never auto-attest.
### When to reject
Any of:
- Any `finding.status === 'fail'` — Worker's claim is contradicted by re-evaluation.
- Any `finding.status === 'inconclusive'` — Worker's evidence is ambiguous or missing.
- Worker's `cleo verify` ledger references a commit SHA unreachable from the validation branch HEAD.
- Required AC evidence missing from Worker's declared atoms (partial ledger).
---
## Max-N Runtime
The shipped runtime is `runValidatorMaxN` in
`packages/core/src/lifecycle/validator/runtime.ts` (T10512). It drives
the canonical Lead → Worker → Validator loop with one shared retry
counter spanning BOTH semantic (REJECT) and infra (timeout, etc.) faults.
### Runtime entry point
```ts
import { runValidatorMaxN } from '@cleocode/core'; // exported barrel TBD
const result = await runValidatorMaxN(
workerTaskId,
{
spawnValidator: async (req) => { /* wraps spawn.validator */ },
respawnWorker: async (taskId, rejection, attempt) => { /* re-dispatch */ },
sleep?: defaultSleep, // injectable for tests
now?: () => isoTimestamp, // injectable for tests
},
{
validatorRetryMax?: 3, // DEFAULT_VALIDATOR_RETRY_MAX
projectRoot?: string, // defaults to getProjectRoot()
suppressAudit?: false, // tests can suppress JSONL writes
},
);
```
### Terminal outcomes
`ValidatorRuntimeResult` is one of three shapes:
| `outcome` | Meaning | Lead action |
|-----------|---------|-------------|
| `'attest'` | Validator returned `ValidatorAttestation` within retry cap | Mark Worker's task done; bindings already written by `validator.attest` |
| `'escalate-hitl'` | Shared retry counter exhausted (transient faults) | Open a HITL approval gate; Lead aggregates `result.attempts` into the spawn prompt |
| `'escalate-permanent'` | First-occurrence permanent fault (e.g. `no-acs`, `tool-not-resolved`, post-downgrade `validator-OOM`) | Route to AC-backfill, project-context.json fix, or infra team — never retry |
The full `result.attempts` array preserves the in-memory audit trail
mirrored to `.cleo/audit/validator-retries.jsonl` (one JSON line per
attempt, fields: `timestamp`, `taskId`, `attemptNumber`, `faultKind`,
`classification`, `retryDecision`, optional `detail`).
### Max-N infra-fault row catalogue
The runtime consumes one row per detected fault and chooses retry
strategy from this canonical table. Rows split into two families:
**semantic faults** (Validator decided but couldn't reach a verdict)
and **infrastructure faults** (Validator process / transport itself
failed before it could decide). Each row specifies retry count,
backoff strategy, and transient-vs-permanent classification — these
values are sourced from the `MAX_N_ROWS` constant in `runtime.ts`.
| Fault | Family | Retry count | Backoff | Classification | Escalation atom |
|-------|--------|-------------|---------|----------------|-----------------|
| `validator-rejected-no-acs` | semantic | 0 | n/a | permanent | `E_VALIDATOR_NO_ACS` → HITL for AC backfill (ADR-066) |
| `validator-partial` | semantic | 1 | immediate | transient on first occurrence | `E_VALIDATOR_PARTIAL` after retry → HITL |
| `validator-unreachable` | semantic | 2 | exponential (10s / 30s) | transient | `E_VALIDATOR_UNREACHABLE` after retries → HITL |
| `tool-not-resolved` | semantic | 0 | n/a | permanent | `E_TOOL_NOT_RESOLVED` → Lead backfills `.cleo/project-context.json` then re-spawns |
| **`timeout`** | **infra** | **2** | **exponential (5s / 30s)** | **transient** | `E_VALIDATOR_TIMEOUT` after retries → HITL. Validator process exceeded `subagentTimeoutSeconds` (default 300s — `DEFAULT_SUBAGENT_TIMEOUT_MS`). A slow LLM call often completes on retry; persistent timeouts indicate a stuck process or pathological task input — investigate input size before further retries. |
| **`conduit-drop`** | **infra** | **3** | **immediate** | **transient** | `E_VALIDATOR_VERDICT_DROPPED` after retries → HITL. The `validator.verdict` message was emitted but lost on the Conduit transport before the orchestrator observed it. Conduit transport latency is low so immediate retry is safe; persistent drops indicate a Conduit subsystem failure (escalate to infra team, not HITL-AC-backfill). |
| **`validator-OOM`** | **infra** | **1** | **immediate, downgraded model tier** (Sonnet → Haiku) | **permanent if downgrade also OOMs** | `E_VALIDATOR_OOM` after retry → HITL. Validator process killed by kernel OOM-killer or hit Node V8 heap limit. The runtime sets `spawnReq.downgradeModelTier = true` on the next attempt; second OOM indicates the task input itself is pathological (gigabyte-scale diff, runaway AC list) — Lead MUST investigate task input size BEFORE further retry. |
**Retry-counter accounting**: each fault consumes ONE slot of the
shared `validatorRetryMax` budget (default `DEFAULT_VALIDATOR_RETRY_MAX = 3`)
regardless of family. Semantic + infra faults share the same counter —
three consecutive faults of ANY mix triggers HITL escalation. This
prevents an infinite-loop adversary where alternating fault kinds bypass
the cap.
**Permanent classification**: a row marked `permanent` short-circuits
the retry loop immediately — the runtime returns `outcome: 'escalate-permanent'`
on the first occurrence without consuming further retries.
### Infra-fault vs reject — the critical distinction
| Symptom | Classification | Why |
|---------|---------------|-----|
| Worker's `tool:test` evidence atom genuinely fails on re-run | **REJECT** (semantic) | The Worker shipped broken code — emit `ValidatorRejection` with `status: 'fail'` |
| `tool:test` fails because `.cleo/project-context.json` is missing or `testing.command` returns "not found" | **INFRA-FAULT `tool-not-resolved`** | The Validator's environment can't evaluate the atom — verdict is not safely binary |
| Worker cited `commit:9f3e2a1` but unreachable from validation branch | **REJECT** (semantic) | The Worker's ledger is broken — emit `ValidatorRejection` citing the bad atom |
| Validator can't reach `npm` registry to install dev deps required by `tool:lint` | **INFRA-FAULT `validator-unreachable`** | Transient network issue, not the Worker's fault |
This distinction matters: rejects loop back to the Worker (productive
feedback via `respawnWorker(taskId, rejection, attempt)`); infra-faults
trigger Validator-only retries (no Worker churn). Mis-classifying an
infra-fault as a reject sends the Worker chasing a phantom bug;
mis-classifying a real reject as an infra-fault burns retry budget
without surfacing the actual defect.
---
## Anti-Patterns
| Pattern | Problem | Solution |
|---------|---------|----------|
| Validator re-uses Worker's agent instance | Violates VAL-001; same blind spots, same incentives | Lead MUST call `spawn.validator` which provisions a fresh worktree-isolated subagent |
| Validator trusts `.cleo/cache/evidence/` cache hits | Worker may have populated the cache with a stale result; the "re-run" is actually a no-op | Cache is disabled by default for Validator runs; explicit opt-in atoms are logged |
| Validator returns "partial pass" | Violates VAL-004 envelope schema; `validatorAttestationSchema.refine` rejects any non-pass finding | Build `ValidatorRejection` whenever any `finding.status !== 'pass'` |
| Validator modifies the worktree to "make tests pass" | Violates VAL-003; you are not the Worker | Emit `ValidatorRejection` with `summary: "tests-fail-on-clean-checkout"` |
| Validator emits a `ValidatorFinding` with empty `reasoning` | Rejected by `validatorFindingSchema` (`min(1)`) | Always populate `reasoning` — for `fail` and `inconclusive`, explain WHY |
| Validator uses `AC<n>` alias as `finding.acId` | `validator.attest` AC-existence check returns `E_VALIDATOR_AC_NOT_FOUND` | Use the UUID from `validator.ac-pull`'s `ac.id` field, NOT `ac.alias` |
| Validator auto-attests on empty `acs[]` result | Violates VAL-007; AC-less task cannot be attested | Surface as `validator-rejected-no-acs` permanent fault; HITL must backfill ACs per ADR-066 |
---
## Relationship to Existing Skills
This skill is the runtime counterpart to two existing skills that
already document the IVTR pipeline statically:
- **ct-validator** (`packages/skills/skills/ct-validator/SKILL.md`) —
static schema / RFC-2119 / document-structure validation at the
LOOM `validation` stage. `cleo-validator` (this skill) is the
DYNAMIC peer-review role that consumes `ct-validator`'s static
findings and adds independent evidence re-execution on top.
- **ct-ivt-looper** (`packages/skills/skills/ct-ivt-looper/SKILL.md`)
— the autonomous Implement-Validate-Test loop the Worker runs.
`cleo-validator` reviews the result AFTER the IVT loop converges;
the loop's `ivtLoopConverged: true` manifest entry is one of the
evidence references the Validator may cite in `finding.evidenceRefs`.
The Lead role (`ct-lead`) orchestrates all three: spawn Worker → IVT
loop → `spawn.validator` → `runValidatorMaxN` → aggregate verdict →
either complete or loop-back via `respawnWorker`.
---
## Cross-references
- **ADR-051** — Programmatic gate integrity (evidence atom grammar the
Validator may cite in `finding.evidenceRefs`)
- **ADR-066** — Acceptance criteria required on every task (precondition
for `validator.ac-pull` returning a non-empty roster)
- **ADR-083 §4** — Cleo persona + hierarchy; defines Validator role tier
- **SG-IVTR-AC-BINDING (T10377)** — owning saga; Council §3.1 action #5
mandates this SKILL ships aligned with the contract
- **T10379** — parent epic (E-VALIDATOR-SKILL-DRAFT)
- **T10383** — sibling epic (E-VALIDATOR-ROLE) that owns the contract +
tools + runtime
- **T10495** — initial SKILL.md draft (PR #769)
- **T10496** — added the infra-fault row catalogue (PR #771)
- **T10510** — shipped the contract types (PR #781)
- **T10511** — shipped the four `validator.*` / `spawn.validator` SDK tools (PR #788)
- **T10512** — shipped `runValidatorMaxN` runtime (PR #787)
- **T10514** — this revision: aligned SKILL.md to shipped reality
- **T10515** — integration test will graduate `metadata.stability` to `stable`
---
## Status
**v2.0.0, stability: experimental**. The contract types, the four SDK
tools, and the Max-N runtime have all shipped. The remaining gap is
the end-to-end integration test (T10515) that drives a real
Lead↔Worker↔Validator loop against `runValidatorMaxN`. Once that test
runs green, `metadata.stability` bumps to `stable`. Until then this
SKILL ships ALIGNED — every signature, error code, and Max-N row in
this document reflects code that exists in `main`.
No comments yet. Be the first to comment!
