Prd

# PRD: 05.5 — the setup skill

- **Kind:** feature
- **Spec:** [.spec/05-autopilot.md](../../.spec/05-autopilot.md) §9 — 05.x series, after [05.4](05.4-cockpit-tab.md)
- **Signed off 2026-09-19** by the PI. Three rulings taken at sign-off, each as recommended:
  the PI **may** override a `crux-design` rejection with their sentence recorded verbatim in
  `## Guidance` (§G); the new `objective-op` lint **refuses** rather than warns (§E); and the
  skill **writes** `auto/<qid>/plan.md` and stops, leaving `crux auto approve` to the PI (§C).
  A fourth ruling reshaped §B: the conversation must not over-ask — three asks, two
  approvals, everything else derived, measured or defaulted.
- **Engine version:** `3.3`, **unchanged**. No vault-format change, no verdict change, no
  roll-up change, no `engine.snapshot` change. No migration. See *Backward-compat* for the one
  place a previously-accepted plan could newly be refused.

## Problem / motivation

Spec §9 names **twenty slots plus a baseline** that a flight plan must carry before
`crux auto run` will touch it, and `flight_plan_problems`
([engine.py:5927](../../skills/crux/scaffold/engine.py)) refuses a plan that is short of any
of them. Today there is nothing that gets a PI from "I want to search this" to a plan that
passes. `templates/flight_plan.md` exists, but it is **not read by any code** — grep finds no
reference to it outside a selftest comment. It is a document an agent is supposed to fill in,
and no instructions exist for filling it in.

That gap is not a convenience problem. Spec §9 closes with the line it calls *"the single most
important line in the plan"*:

> **The objective must come from a check that discriminates against the null.** Optimizing a bar
> the hypothesis was always going to clear is the machine-scale form of the bias spec 09 exists
> to prevent: the search will clear it, thousands of times, at cost, and teach nothing.

The engine enforces the *shape* of that — `flight_plan_problems`' `discriminates` check
([engine.py:6220](../../skills/crux/scaffold/engine.py)) refuses a plan whose objective address
is not named by a verifiable marked `discriminates:: true`. It cannot enforce the *order*. A PI
who picks the number first and writes a check around it afterwards produces a plan that passes
every lint and is exactly the failure the line forbids. **Only a conversation can get the order
right**, and that conversation is this slice.

Second, the template's `## Verifiables` block is a stub:

```
- [ ] <<verifiable>>
      fails-if:: <<fails_if>>
      discriminates:: true
- [ ] [outcome-neutral] <<control>>
      fails-if:: <<control_fails_if>>
```

`<<verifiable>>` has to satisfy four independent engine rules at once — `check-grammar`
(it must begin `<key.path> <op> <number>`), `discriminates` (it must name the objective
address), `control` (the set must carry an outcome-neutral one) and `scenario` (each needs a
failure scenario). A placeholder that says `<<verifiable>>` teaches none of them. **Defining
these placeholders is this slice's own deliverable**, per the 05.5 work item.

Third, a trap this project has already hit once: a hard-coded `<=` in a placeholder is **wrong
for `direction: max`**, and 05.2 reverted exactly such a change. The engine knows the rule —
`auto_scored_pass` is `value >= bar if direction == "max" else value <= bar`
([engine.py:6530](../../skills/crux/scaffold/engine.py)) — but nothing pins the objective's
*check text* to the objective's *direction*. The demo fixture happens to get it right
(`obj.score >= -0.5`, `direction:: max`); a plan that wrote `obj.score <= -0.5` under
`direction:: max` passes every lint today and searches against its own bar.

## Design decision

### A · One skill, at `skills/crux-autopilot/`

A skill, not an agent. Agents (`agents/<name>/AGENT.md`) are isolated single-shot graders;
this is a multi-turn conversation with the PI that *spawns* three of those agents. It matches
`crux-cockpit` — a skill that drives crux verbs on the PI's behalf and hands back one artifact.

`install.sh` discovers skills by scanning `skills/`
([install.sh:6](../../install.sh): *"Drop a new skill folder under skills/ and it's installed
automatically"*), so **no roster edit is needed** and the agent roster in
`selftest.run_agent_roster` is untouched — this slice adds no agent.

`skills/crux/SKILL.md` gains **one cross-reference line** pointing at the new skill from the
`auto check` / `auto approve` rows of its verb table. That is the whole edit, and it carries
the five-fixture re-pin obligation (see *Build notes*).

### B · The conversation asks only what it cannot derive, measure or default

**PI, 2026-09-19:** the setup conversation must be fast and to the point — it must not
over-ask, it must not over-engineer, it must sort out the most important things and **leave the
details and the non-obvious choices to the autopilot to figure out.** That ruling is the design
of this section, not a note on its tone.

Spec §9 names twenty slots. **Only three of them are asks.** The rest are derived from an
answer already given, measured by a command, or taken from the template default and stated back
in one line.

| # | the skill's turn | why it cannot be skipped |
|---|---|---|
| 1 | **ask** — the anchor question, and the goal in one falsifiable sentence | the frame is the PI's; nothing can derive it |
| 2 | **approve** — the null, proposed by `crux-null` | a ruling, not a value; spec 05 requires the PI's yes |
| — | *derived* — the checks, their kinds, the combination rule, the control | `crux-verifiables`, against the approved null |
| — | *derived* — the objective address | **read off** the discriminating check just written |
| 3 | **ask** — which direction is better, and the bar | see below; direction is the one thing that must never be guessed |
| 4 | **ask** — the harness, in one turn: scorer command, run command, frozen paths, writable paths, budget ceiling | only the PI knows their repo |
| — | *measured* — the baseline | `crux auto check` dry-runs the scorer and prints it (§H) |
| — | *defaulted* — the remaining fourteen slots | template defaults, stated back, not asked |
| 5 | **approve** — `crux-design`'s verdict, then `crux auto approve` | the PI's signature (§C, §G) |

**The objective is read off a check, never chosen.** Act 3's ask is the bar and the direction,
*after* `crux-verifiables` has written the checks — so the address comes out of a check that
already exists, and the engine's `discriminates` lint passes as a consequence rather than as a
hurdle. Spec §9's most important line becomes structurally true instead of a reminder somebody
has to remember. **This ordering survives the shortening; it is the one thing in the
conversation that is load-bearing.**

**Direction is asked, never inferred.** A metric named `loss` is usually minimised and a metric
named `score` usually maximised, and "usually" is exactly how 05.2's reverted `<=` got written.
One short question costs a turn; a backwards bar costs the whole run.

The fourteen defaulted slots are `mode`, `islands`, `island_cap`, `parallel_total`,
`parallel_island`, `retries`, `retention`, `replicates`, `rule`, `steward` / `steward_every`,
`stall_attempts`, `abort_invalid_runs`, `scorer_timeout`, and the agent block
(`agent`, `agent_failover`, `agent_cooldown`, `agent_probe`, `agent_probe_timeout`). The skill
states them in **one line**, not fourteen, and moves on.

**Five rules on the skill's voice, each greppable in `SKILL.md`:**

1. **One question per turn.** Never a questionnaire, never a numbered list of choices.
2. **Never ask what a command can measure.** The baseline is scored, never asked for.
3. **Never ask what the template defaults.** State the default in passing; discuss it only if the
   PI raises it.
4. **Never ask the PI to choose between options that differ only in something the search itself
   would discover.** That is what the autopilot is for — a choice the bandit can make in ten
   attempts must not cost a turn of the PI's attention.
5. **"I don't know" takes the default.** The skill records which slot was defaulted that way and
   moves on; it never escalates, re-asks, or explains the option space.

The skill never opens a turn by summarising the previous one, and it never closes one by listing
what remains. Those two habits are what make a five-slot conversation read as twenty.

### C · The skill writes the plan file. It never signs it.

`crux auto approve` is the PI's signature (spec 05.2, and `skills/crux/SKILL.md:353` marks it
◆ — *"Never run this without the PI's yes"*). The skill:

1. **writes** `auto/<qid>/plan.md` from the template,
2. runs `crux auto check` and fixes what it reports,
3. spawns `crux-design`,
4. **stops**, and hands the PI the path plus the exact `crux auto approve` line to run.

Writing the file is not approving it: `flight_plan_hash`
([engine.py:6541](../../skills/crux/scaffold/engine.py)) means the approval stamp covers a hash
of the document, so any edit outside `## Guidance` clears it. A written-but-unapproved plan is
inert — `crux auto run` refuses it. So the skill writing the file costs the PI nothing and
saves them transcribing twenty slots by hand.

The skill also never runs `crux auto guide` on the PI's behalf: `## Guidance` is the PI's own
words, append-only (`SKILL.md:351`).

### D · The `## Verifiables` placeholders become a worked grammar

The stub is replaced by placeholders that carry the four rules in their own shape, with the
operator **derived, never literal**:

```
## Verifiables

- [ ] <<address>> <<bar-op>> <<bar>> — <<what reaching this number would mean>>
      fails-if:: <<the world in which this number is reached and the goal is still false>>
      discriminates:: true
- [ ] [outcome-neutral] <<control-key>> <<control-op>> <<control-number>> — <<what this rules out>>
      fails-if:: <<the world in which this control passes and the run is still invalid>>
```

with a legend directly under the block, inside the template, stating:

- `<<bar-op>>` is **`>=` when `direction: max`** and **`<=` when `direction: min`**. It is never
  written as a literal in the template, because 05.2 reverted exactly that.
- the first check **must** repeat `<<address>>` verbatim — that is what `discriminates` matches
  on, token-wise ([engine.py:6221](../../skills/crux/scaffold/engine.py)).
- `<<bar>>` is the same number as `bar::` in `## Objective`.
- the control's operator is **not** derived from direction: a control is outcome-neutral, so it
  may point either way. The demo's `obj.score <= 0` under `direction: max` is correct.
- `fails-if::` is the world where the check passes and the claim is still false, never a
  restatement of the check.

### E · One pure helper and one new lint: `objective-op`

`engine.auto_direction_op(direction)` → `">="` for `max`, `"<="` for `min`, raising `CruxError`
otherwise. Pure, total, ten lines, beside `AUTO_COMPARISON_OPS`
([engine.py:5703](../../skills/crux/scaffold/engine.py)).

`flight_plan_problems` gains one check, `objective-op`: **the discriminating verifiable that
names the objective address must use the operator that agrees with `direction`.** It fires only
on the check the existing `discriminates` rule already singles out; controls and every other
check are untouched.

This is the only place in the slice where a plan that is accepted today could be refused
tomorrow — see *Backward-compat*. **It refuses** (PI, 2026-09-19).

### F · `closer: true` in the template, not in the engine

The 05.5 work item makes `closer: true` the default *in practice*, by writing it into new
plans. `templates/flight_plan.md`'s frontmatter changes `closer: false` → `closer: true`.

**`fm.get("closer")`'s own default stays falsey in the engine**, so a plan written before this
release — which carries `closer: false` explicitly, or omits it — runs exactly as it did. This
matters beyond tidiness: the `check-grammar` rule is skipped when `closer` is true
([engine.py:6205](../../skills/crux/scaffold/engine.py)), because with a closer the prose checks
are `crux-close`'s to propose. Flipping the engine default would silently relax that lint for
every old plan. Flipping the template does not.

### G · When `crux-design` rejects

`crux-design` detects three causes of a partial answer and fixes only the third (its own
definition: *"Detects all three causes … fixes the third, and hands the other two to the agents
that own them"*). The skill's rule, by cause:

| cause | who owns it | what the skill does |
|---|---|---|
| compound claim | act 1 | reopens act 1; the goal is re-split, and acts 2–4 are redone against the new frame |
| a check that does not follow from the claim | act 3 | reopens act 3; `crux-verifiables` runs again against the same approved null |
| the run cannot discriminate | `crux-design` itself | takes its fix, then re-runs `crux auto check` |

**The skill never proceeds past a rejection on its own, and it never edits `crux-design`'s
verdict.** Only the PI may say "go anyway"; if they do, the skill records their sentence
verbatim in `## Guidance` via `crux auto guide` — the PI's words, on the PI's yes — so the run's
own transcript carries why. (PI, 2026-09-19.)

### H · The baseline is scored, not asserted

Spec §9: *"Setup ends by scoring the current code and recording it."* Act 5's exit is
`crux auto check` passing, which already dry-runs the scorer
(`SKILL.md:340`). The skill records the number it printed as `baseline:` rather than asking the
PI for one, and states the number back to them. Terminal hypotheses already under the anchor are
adopted as further starting observations, per §9.

## Scope — what this does not do

- **No change to any of the three agents.** Byte-identical, asserted.
- **No new `crux` verb.** The skill drives the verbs that exist.
- **No change to `closer:`'s engine default**, to `AUTO_COMPARISON_OPS`, to `auto_check_comparison`'s
  grammar, or to how a tick is derived from a number.
- **No autonomous approval.** `crux auto approve` and `crux auto run` stay the PI's.
- **Does not address spec §8's run-branch commits** — still deliberately unassigned (2026-09-18).
- **No tier-1+ validation.** That is 05.6.

## Acceptance criteria

Each is a command with an expected result. `SUITE` below means
`./crux selftest 2>&1 | grep '^  FAIL ' | grep -cv 'a stale install.sh'` → `0`.

1. `test -f skills/crux-autopilot/SKILL.md` → exit 0.
2. `bash install.sh` with `SKILLS_DIR` set to a temp dir installs `crux-autopilot` alongside
   `crux`, `crux-wiki`, `crux-cockpit` — no install.sh edit required.
3. `git diff origin/main -- agents/` → **empty**. The three agents are unchanged.
4. `python3 -c "import engine as E; print(E.auto_direction_op('max'), E.auto_direction_op('min'))"`
   → `>= <=`; `auto_direction_op('sideways')` raises `CruxError`.
5. A plan with `direction: max` whose discriminating check reads `obj.score <= -0.5` is reported
   by `crux auto check` with the `objective-op` problem naming both the direction and the
   expected operator. It is a refusal, not a warning.
6. The same plan with `>=` reports no `objective-op` problem.
7. The demo fixture `_loop_repo`'s plan still passes `crux auto check` unchanged — its
   `obj.score >= -0.5` / `direction:: max` pair already agrees.
8. `grep -c '<=' skills/crux/scaffold/templates/flight_plan.md` counts **no** occurrence inside
   the `## Verifiables` placeholder block — the operator is named by derivation, not written.
9. `grep '^closer:' skills/crux/scaffold/templates/flight_plan.md` → `closer: true`.
10. `python3 -c "import engine as E; ..."` on a plan omitting `closer:` still applies
    `check-grammar` — the engine default is unchanged.
11. A `setup-01` eval fixture scores a proposed plan against a planted key: the objective address
    is read from a check written before it, the operator agrees with direction, and a control is
    present. No model, no network, no vault read.
12. `SKILL.md` carries all five voice rules of §B as greppable lines, and names exactly
    **three asks** (frame, direction+bar, harness) and **two approvals** (null, design +
    `auto approve`). A grep for the fourteen defaulted slot names finds them in **one**
    line, not fourteen — the conversation must not re-open a defaulted slot.
13. `SUITE` → `0`, on all eight CI jobs, Windows included.
14. Suite growth: the new sections add **under 3 s** to the ~72 s base, measured on an idle main
    checkout. The number goes in the PR.
15. An existing vault written before this slice loads unchanged: `crux auto status` on
    `~/crux-autopilot-demo/cruxvault` prints the same four numbers as before the merge.
16. `ENGINE_VERSION` is still `"3.3"`.

## Backward-compat / migration

No vault-format change and no migration. One behaviour change, confined to §E: a flight plan
whose **discriminating** check uses the operator that disagrees with its own `direction` is
newly reported by `flight_plan_problems`. Such a plan is, by construction, searching against a
bar it reads backwards — the defect §9's most important line exists to prevent — so reporting it
is the point. Every plan in the repo today (the tier-0 fixture, the template once filled) already
agrees. The report is a **refusal** (PI, 2026-09-19): a plan that reads its own bar backwards
must not start.

`closer:`'s engine default is untouched, so a 05.2-era plan runs byte-identically (§F).

## Build notes

- **`skills/crux/SKILL.md` is edited**, so the five
  `skills/crux/evals/fixtures/persona-01/submissions/{clean,id-led,idle,mirror,notebook}.json`
  must be re-pinned **in the same chunk**: `agent_sha` = `shasum -a 256 skills/crux/SKILL.md`
  (**run it; never copy an old value**), plus a dated `RE-PINNED …` clause on each `_note`.
  Otherwise the persona eval returns REFUSED and the suite dies with `KeyError: 'hard'`.
- **The engine stays pure.** `auto_direction_op` and the `objective-op` check are pure functions
  on strings — no process, no git, no threads.
- **Python 3.8 is the CI floor.** No `dict | dict`, no `list[str]`, no `match`, no
  `functools.cache`.
- **`grep FAIL` on selftest output lies.** Use the `SUITE` form above.
- **Fixtures name only commands the suite already depends on** — `python3`, never `claude`.
- `CHANGELOG.md` entries go **newest-first** under `## [Unreleased]`.