Test Readiness

---
name: test-readiness
description: >
  Orchestrates the full test-readiness pipeline (phases 1–5), once per
  qualifying repo under repos/, each in its own worktree + branch. Runs
  test-inventory → test-design → test-migration → test-roadmap →
  test-fix in sequence per repo, starting from the first stale artifact.
  Invoked by the `shipwright-test-readiness` cron on a daily schedule; exits
  early for a repo when all of its phase artifacts are fresh.
---

# test-readiness skill

## Purpose

Drive the complete test-readiness pipeline from the first stale phase to
publication. This skill is the cron-facing entry point — it picks up where the
staleness guard left off and runs all necessary phases to bring the docs current.

## When invoked

By the `shipwright-test-readiness` cron, which runs
`/shipwright:test-readiness --full --publish` daily. The cron fires
unconditionally on schedule; the flags below decide what actually happens.

## Arguments

Flags are additive — invoking with no flags preserves the original behavior
(staleness-gated, phase 5 in dry-run), so agents still on the old cron prompt
keep working unchanged.

| Flag | Effect |
|------|--------|
| `--full` | Skip the staleness check entirely and run all phases 1→5 regardless of artifact freshness. Without it, the staleness gate decides the starting phase and exits early when everything is fresh. |
| `--publish` | Run phase 5 as a **real publish** — forward real-queue mode to `/shipwright:test-fix` so it writes task-store tasks unattended (no confirmation gate — test-fix never has one, matching entropy-fix/error-fix). Without it, phase 5 runs `--dry-run` (preview only, no task-store writes). |
| `--dry-run` | Explicit preview: phase 5 runs `--dry-run`. This is the default when `--publish` is absent. |

## Staleness check

**Skipped entirely when `--full` is passed** (the cron's default) — all phases
1→5 run. The check below only applies when `--full` is absent.

Before running any phase, re-read the phase artifacts (scoped to the current
repo/worktree — see Step 1) to determine the earliest stale phase. The
artifacts and their canonical paths (relative to the repo root):

| Phase | Artifact path |
|-------|--------------|
| 1 — test-inventory | `docs/test-readiness/test-inventory.md` |
| 2 — test-design | `docs/test-readiness/test-system.md` |
| 3 — test-migration | `docs/test-readiness/test-migration.md` |
| 4 — test-roadmap | `docs/test-readiness/test-readiness-plan.md` |

Threshold: 24 hours (default). If the file is missing or older than 24 hours,
it is stale. All phases from the first stale one through phase 4 run; phase 5
(publish) always runs when phase 4 ran, because the roadmap needs to be
re-published after any update.

## Process

This agent's workspace can have multiple repos checked out under `repos/`
(see `plugins/shipwright/scripts/check-helpers.ts`'s `resolveRepoDirs`).
Step 1 below is repo-aware: it resolves a list of qualifying repos and Steps
2-3 run once **per repo**, each in its own worktree + branch, independent of
every other repo. Step 4 (Report) prints one aggregated summary across every
repo processed.

### Step 1: Resolve the repo list

Determine which repos to process, in this priority order:

1. **Precheck-driven (preferred).** This cron's `preCheck`
   (`shipwright:check-test-readiness.ts`) already iterated every repo under
   `repos/`, applied the opt-in qualification (a repo qualifies only if it has
   a `docs/test-readiness/` directory — a repo without one is not the implicit
   target and is skipped), and its stdout — which became this prompt (see
   the agent type manifest's cron `preCheck` contract: "When a preCheck script is
   set, its stdout becomes the actual prompt sent to Claude") — lists exactly
   which repo(s) have a stale or missing phase artifact, one repo name
   (`org/repo`) per section. Parse the repo names out of the invoking prompt
   and use that as the repo list. Skip any repo not named in the precheck
   output — it had nothing to run.
2. **Fallback (manual invocation, or no repo list available in the prompt).**
   Iterate `repos/*` directly and keep only repos with a `docs/test-readiness/`
   directory (the same opt-in signal the precheck uses):
   ```bash
   for dir in repos/*/; do
     [ -d "$dir/.git" ] && [ -d "$dir/docs/test-readiness" ] && basename "$dir"
   done
   ```
   A repo under `repos/` with no `docs/test-readiness/` directory is skipped
   cleanly — it is never silently treated as the implicit single target.

For each resolved repo, match the precheck's `org/repo` name back to its local
clone directory `repos/{dirname}` by checking each `repos/*/`'s
`git remote get-url origin` (or `.git/config`) for that owner/repo — same
matching approach as `research-docs.md` Step A0.

**For each repo in the resolved list, set up (or reuse) a worktree + branch
and run Steps 2-4 there** — never edit `repos/{dirname}` directly (see this
repo's own root `CLAUDE.md` for the worktree convention).

**Open-PR supersession check (run first, before any branch is created).**
A prior day's docs-refresh PR touching the same `docs/test-readiness/*.md`
files may still be open — e.g. stuck on review, CI, or awaiting a human.
Branching fresh from `origin/main` every day regardless creates a second PR
competing for the same files; if the new one merges first, the stuck PR
becomes permanently unmergeable (confirmed: a same-pattern collision turned
an older open docs-refresh PR CONFLICTING once a newer one merged). Before
creating a new branch, query for an existing **open** PR on any
`docs/test-readiness-refresh-*` branch (any date, not just today) for this
repo:

```bash
gh pr list --repo {org}/{repo} --state open \
  --json number,headRefName,createdAt
```

`gh` has no glob support for head-branch matching, so filter the JSON result
client-side, keeping only entries whose `headRefName` starts with
`docs/test-readiness-refresh-`. This check is distinct from, and runs
**ahead of**, the same-day-reuse check below — it looks for *any* earlier-day
open PR, not just today's.

- **No matching open PR found:** fall through to the fresh-branch flow below,
  unchanged.
- **Exactly one earlier-day open PR found:** reuse that PR's branch instead
  of branching fresh. Check out the existing branch into a worktree (or reuse
  an existing worktree for it if one is already present) and merge current
  `origin/main` into it:
  ```bash
  git -C repos/{dirname} pull
  git -C repos/{dirname} worktree add \
    $SHIPWRIGHT_WORKTREE_DIR/{dirname}-docs-test-readiness-refresh-{existing-date} \
    {existing-branch-name}
  cd $SHIPWRIGHT_WORKTREE_DIR/{dirname}-docs-test-readiness-refresh-{existing-date}
  git merge origin/main
  ```
  - **Merge succeeds cleanly:** proceed to Steps 2-4 in this worktree as
    normal — the branch and its open PR are now caught up with `main` and
    will absorb this run's updates instead of being superseded by a
    competing PR.
  - **Merge conflict:** do not attempt to resolve it automatically. Abort the
    merge (`git merge --abort`), **skip Steps 2-4 for that repo only**, and
    report the conflict explicitly and distinctly in Step 4's aggregated
    summary (see Step 4 below) — do not fold this into the "skipped — all
    artifacts fresh" case, since the repo was not fresh, it was blocked.
    Other repos in the resolved list are unaffected and continue processing
    normally.

**Fresh-branch flow (no earlier-day open PR found, or same-day rerun):**

```bash
git -C repos/{dirname} pull
git -C repos/{dirname} worktree add \
  $SHIPWRIGHT_WORKTREE_DIR/{dirname}-docs-test-readiness-refresh-{YYYYMMDD} \
  origin/main -b docs/test-readiness-refresh-{YYYYMMDD}
```

Branch naming: `docs/test-readiness-refresh-<YYYYMMDD>` (today's date). If a
worktree/branch with that name already exists (e.g. a same-day rerun), reuse
it rather than recreating — check `$SHIPWRIGHT_WORKTREE_DIR` for an existing
`{dirname}-docs-test-readiness-refresh-{YYYYMMDD}` directory first. This
same-day-reuse check is separate from the open-PR supersession check above:
this one matches on today's exact branch name only, regardless of PR state;
the supersession check above matches on any earlier-day PR still open on
GitHub.

All relative paths in Steps 2-4 (`docs/test-readiness/*.md`, phase artifact
mtimes, git history, etc.) resolve against that repo's worktree, not the bare
clone under `repos/`. Commit each repo's phase results (in its own worktree,
on its own branch) before moving to the next repo — one repo's work must not
block or be blocked by another's.

### Step 2: Determine the starting phase

If `--full` is set, start at phase 1. Otherwise check mtime of each phase
artifact in order (1 → 4), scoped to the current repo's worktree; the first
artifact that is missing or older than 24 hours marks the starting phase, and
if all are fresh for this repo, skip straight to the next repo in the
resolved list (no phases run for this repo).

### Step 3: Run phases sequentially

From the starting phase through phase 5, in the current repo's worktree:
- Phase 1: `/shipwright:test-inventory`
- Phase 2: `/shipwright:test-design`
- Phase 3: `/shipwright:test-migration`
- Phase 4: `/shipwright:test-roadmap`
- Phase 5: `/shipwright:test-fix` — pass no extra flag (real queue) when
  this skill was invoked with `--publish`; otherwise pass `--dry-run`
  (preview only, no task-store writes).

Run each skill using the Skill tool. If any phase fails for this repo, report
the failure for that repo and stop running further phases **for that repo
only** — do not run subsequent phases for it, as each depends on the previous
artifact. A failure in one repo does not stop the remaining repos in the
resolved list; move on to the next repo after logging the failure.

After phase 5 (or an early stop due to failure) completes for the current
repo, return to Step 1's per-repo loop for the next repo in the resolved
list. Once every repo has been processed (run or skipped for being fresh),
proceed to Step 4.

### Step 4: Report

After every repo in the resolved list has been processed, print one
aggregated summary across all repos, with a per-repo section: which phases
ran (or "skipped — all artifacts fresh" / "skipped — precheck did not flag
this repo" / "skipped — reuse-merge conflict, needs manual resolution" for
the open-PR supersession case above), and the number of task-store tasks
created or updated (from phase 5 output) for that repo. The reuse-merge
conflict status is reported distinctly from "skipped — all artifacts
fresh" — the two mean different things (blocked vs. nothing to do) and must
not be folded together, so a human scanning the summary can immediately spot
the repo that needs manual conflict resolution on its open PR.

## Failure handling

If a phase fails:
- Log which phase failed and include any error output.
- Do not run subsequent phases.
- Exit with a clear summary so the cron log shows the failure.
- A failure in the skill itself is always logged.

## Notes

- This skill handles its own staleness check (step 2) **unless `--full` is
  passed**. With `--full` (the cron default) every phase runs; without it, a
  fresh set of artifacts makes the skill exit early.
- Phase 5 (`test-fix`) has no confirmation gate — the cron passes `--publish`,
  which forwards real-queue mode so it writes task-store tasks unattended.
  The publish step is idempotent (it dedupes on already-active T-NNN
  task-store IDs per repo), so a daily real run queues tasks for new work —
  including doc-layer tasks — without creating duplicates.