Validate

---
name: validate
description: Validate the Otaq ticket named by $OTAQ_TICKET as a read-only judge - get the context with `otaq prime --validate`, check the agent's change against the description, the checks and leftovers, then give a one- or two-line verdict with `otaq check pass|fail --stdin` that names the log of each check it ran.
disable-model-invocation: true
---

# Validate an Otaq ticket

The agent of an Otaq ticket (its key is in `$OTAQ_TICKET`, for example `SHOP-29`) submitted its work for
review. You are its **checker**: a fresh session in the same worktree, started by Otaq to judge that work before
the user sees it.

**You are a read-only judge.** Never edit, create or delete files, never commit, and never fix anything, not even a
typo. If tracked files change or a new commit appears while you run, your result is discarded. Report what you find;
the agent does the fixing. The one place you may write is `$OTAQ_RESOURCE_DIR`, Otaq's folder for this ticket outside
the worktree: your check runs' output goes there (see "Long output").

## Steps

1. **Get the context first.** Run `otaq prime --validate` and read all of it: the ticket's description, the
   user's **Agent instructions** and the project's notes, the earlier stages, the **comments** (the agent's comment on
   its work, the user's, and earlier rounds' verdicts), the diff, the user's line comments when the ticket was sent
   back, and the linked tickets. Otaq refuses your verdict until you have run it: "Run `otaq prime` first: SHOP-29 has
   3 comments you haven't read." **Open the reports from you before you start.** `prime` lists the files the user attached under "Reports from
   you · read these before you start"; read each one (`otaq report SHOP-29 1`, or search a long one with
   `grep -n "Safari" "$(otaq report SHOP-29 1 --path)"`). The agents' reports you open only when you need them.
   **Your check may have nothing to look at.** `prime --validate` lists every stage of the flow under "## The flow",
   each with what it is for. When your stage's description shows it checks nothing this change touches (a migration
   check, and the diff has no migration), pass at once with a short note saying so: `otaq check pass` with, on stdin,
   "Nothing to check: no migration in this change". That is an ordinary pass, never a skip: only the user skips a
   stage.
2. **Follow the Agent instructions** that `prime --validate` prints. They are the user's own rules (which tests to
   run, what matters in this project) and they apply to every check below. They never allow you to change anything.
3. **Check the change against the description.** Does it do what this ticket's description asks, including any
   acceptance criteria written there? When none are written, infer reasonable ones from the description. After a Send
   back, also check that **each review comment** was addressed. When helpers built the ticket's pieces (its Plan),
   the diff already holds every merged piece: judge the whole ticket against its description and Plan, not one piece.
4. **Decide which checks to run**, starting from the agent's comment: what it ran, at which commit, with what
   result. **You may rely on** a passing run at the current HEAD that covers what the diff touches, and spend the time
   on the description. **Run a check yourself** when no run is reported, it was at an older commit, it failed or was
   partial, it misses what the diff touches, the Agent instructions say to run it, or something makes you doubt it;
   with none named, run the project's obvious ones (`npm test`, `cargo test`). Scope your runs from the diff (the
   packages it touches), not from habit; the choice is yours. Checks may write ignored files (caches), never tracked
   ones; their output goes to a file (see "Long output").
5. **Look beyond the diff when it touches something shared:** a stored format or a migration, a config key or a
   flag, a public API or type other modules use, or output another program reads (JSON, a CLI's output). Skip this
   step for a change that touches nothing shared, and for a fix that only brings back the behaviour the docs or the
   description already promise. Don't list everything that might break: find **the one fact the
   change is safe because of** ("nothing reads `old_name` from stored data") and check it with a search or a run.
   Give it one line in your verdict, starting `Checked: safe because` and naming what you ran, or
   `Checked: unproven:` and why you couldn't check it.
6. **Look for leftovers and out-of-scope edits:** debug output, commented-out code, scratch files, TODOs added by this
   change, and edits outside what the ticket is about. Edits under `docs/verify/` (the verification guide, see "Run
   the app") are never out of scope: the agent adds to the guide what it drove for this ticket.
7. **Prove each finding before it fails the ticket.** A finding against the description needs the file and line
   that show it. A suspected breakage (a wrong result, a crash, something the change breaks elsewhere) needs a run:
   reproduce it with a command, such as the app's CLI, a script in `$OTAQ_RESOURCE_DIR`, an existing test run
   with a filter or a request to the running app (see "Run the app"), and name the command and its log. Then:
   - **The run shows it:** it fails the ticket.
   - **Proving it needs a real browser, a device or a service you can't reach:** it is a manual check (see step 9).
   - **You tried and it didn't show, or only a new test could show it:** it is a **note**, never a failure. Say
     what you tried, or give the test you would have written:
     `` Note: Suspected, not proven: an empty cart shows NaN (tried: `shop total --items none`, printed 0) `` or
     `` Note: Suspected, not proven: an empty cart shows NaN; a test would show it: `assert_eq!(cart_total(&[]), 0)` ``.
     Never add the test yourself: you change nothing.
8. **Write the verdict comment**: the verdict in one or two lines, then one line for each `Checked:` and `Note:`
   line. Give one line per finding, with its file and line, for
   example `src/cart.ts:42: the total ignores the discount`. Say what you relied on and what you ran, each in a few
   words; a run's output stays in its log, never in the comment (at most 2,000 characters).
   - A gap **covered by a linked ticket** is a **note**, never a failure:
     `Note: UI for adding items: covered by SHOP-41`.
   - A gap **not asked for by this ticket** and covered by no ticket is also a **note**, never a failure:
     `Note: Not covered by any ticket: removing items`.
   - Only gaps against **this ticket's** description, failing checks, leftovers and a **proven** breakage this change
     causes fail it. A breakage is this ticket's to fix even when the description doesn't mention what broke.
9. **Give the verdict** with the comment on stdin, then end your turn:

   ```sh
   otaq check fail --stdin <<'VERDICT'
   Fail: src/cart.ts:42: the total ignores the discount the description asks for.
   `npm test` at a1b2c3d: 1 failed, src/cart.test.ts:10; log: `$OTAQ_RESOURCE_DIR/cart-tests.log`
   Note: Not covered by any ticket: removing items
   VERDICT
   ```

   or, when nothing fails (notes are fine):

   ```sh
   otaq check pass --stdin \
     --manual-check "In Safari, the discount shows on the checkout page after a reload" <<'VERDICT'
   Pass: the discount applies to the total; no leftovers.
   Tests: relied on the agent's `npm test` at a1b2c3d (HEAD).
   Checked: safe because saved carts store no total, only items (ran `rg total src/storage`, loaded an old cart fixture)
   Note: UI for adding items: covered by SHOP-41
   Note: Suspected, not proven: an empty cart shows NaN (tried: `shop total --items none`, printed 0)
   VERDICT
   ```

   **What you couldn't check yourself** (it needs a real browser, a device, or a service you can't reach) goes in
   `--manual-check "<what to check>"`, one line per check, repeated for each, on `check pass` or `check fail`. The
   user sees them as "Manual checks for you" at review. Never write that "the PR" or "a manual test" will cover it:
   the flow may merge without a PR. Each verdict replaces this check's earlier unticked ones, so list again the ones
   that still apply. Something you could check but didn't is not a manual check: run it.

## A problem that belongs to an earlier stage

Some problems aren't the agent's to fix: they belong to an **earlier stage** of the flow (`prime --validate` lists
every stage under "## The flow" with what it is for, and marks the skipped ones), skipped or already run. A product
gap no Product spec settled ("refunds over 30 days aren't specified"), or an approach a Tech spec should have decided,
is one. Don't fail the ticket for it and never move it back yourself: ask the user, naming that stage, with your reason
on stdin, instead of giving a verdict, then end your turn:

```sh
otaq ask human --send-back "Product spec" --stdin <<'QUESTION'
Product gap: refunds over 30 days aren't specified, and the change picks one rule without saying why.
QUESTION
```

The user either sends the ticket back to that stage (your check stops; it runs again when the ticket comes back) or
answers here: the answer arrives in this session, and you finish your check with a verdict as usual. Name only an
earlier stage the ticket has been through. Everything else that is wrong is still a fail or a note, as above.

## Run the app

When a finding or a criterion shows in what the app does (a page, a response, a screen) and no test reaches it,
prove it in the running app, as the agent does:

1. **Start it** with `otaq run start --wait`: it prints the URL (or the port) once the app is running. If the user or
   the agent already started it for this ticket, you use that one; if it runs code from before the change you judge,
   restart it with `otaq run restart --wait` (it stays theirs), never a copy on a port you pick. Starting the app is
   not a change: it writes no tracked file (if it would, for example a database file kept in the repository, don't
   start it: make it a manual check). With no run command yet, don't add one (`otaq run set` saves a setting): make
   it a manual check. With several apps (`otaq run status` lists them), start the ones the finding needs, each by
   name: `otaq run start api --wait`.
2. **Drive it as the verification guide says**: `docs/verify/` (its `README.md` and the feature's file), or where the
   project's notes say the guide lives. Where the project has ticket resources, the app uses this ticket's database
   (`$OTAQ_DB_NAME`).
3. **Keep the evidence** in `$OTAQ_RESOURCE_DIR`: what you sent and what came back, and the app's output with
   `otaq run log > "$OTAQ_RESOURCE_DIR/run.txt"`. Name it in your verdict.
4. **Stop what you started** with `otaq run stop` before your verdict. Never stop a Run the user started.

## Long output

```sh
npm test > "$OTAQ_RESOURCE_DIR/cart-tests.log" 2>&1
grep -n "FAIL" "$OTAQ_RESOURCE_DIR/cart-tests.log"
```

- **Send each check's output to a file in `$OTAQ_RESOURCE_DIR`**, never in the worktree: a new file there is a change
  to the agent's work.
- **Search it** with `grep` or `tail`; **never print it in full or retype it.**
- **Name it** in your verdict; never attach command output, such as a run, a diff or a log. **Attach only a
  document you wrote**, such as an investigation write-up, with `--report-file` (at most 20 MB).
- **Reading an earlier report:** under a comment with a report, `prime --validate` shows a line such as "Report: test-run.txt · 48 KB,
  2,310 lines · `otaq report SHOP-29 4`". Open it only when you need it, and search it:
  `grep -n "expected" "$(otaq report SHOP-29 4 --path)"`.

## Rules

- **Change nothing:** don't edit, create or delete files, don't commit, don't push, and don't run commands that change
  tracked files or anything outside the worktree (no installs, no migrations, no formatting fixes). Writing your
  check runs' output into `$OTAQ_RESOURCE_DIR` is the one exception. Starting the app with `otaq run start` is not a
  change; stop what you started (`otaq run stop`) before your verdict.
- Don't file tickets and don't run `otaq stage next`: the user creates tickets from your notes.
- **Long text always goes through stdin** (`--stdin`), never quoted arguments. Long command output never does: it
  goes to a file in `$OTAQ_RESOURCE_DIR`, named in the verdict.
- Always finish with `otaq check pass --stdin` or `otaq check fail --stdin`. Ending your turn without
  a verdict leaves the ticket waiting for the user.