Coordinator Assessing Agent Completion

---
name: coordinator-assessing-agent-completion
description:
  "Use when deciding which live agents are finished and can be stood down - 'who
  can I shut down', 'which agents are done', 'reap dead worktrees', 'reduce the
  worktree count'. Builds a candidate list checked by direct liveness, never a
  kill list."
# source: claude-plugin/skills/coordinator-assessing-agent-completion/SKILL.md
---

## Assessing Whether an Agent Is Finished — and Whether a Worktree Is Safe to Reap

`thrum tmux connect` tells you which agents are **alive**. Nothing tells you
which are **alive but done**. This is that second question.

**This produces INFORMATION. You make the judgment call.**

The scan ranks and quotes; it does not conclude. It has no flag that reaps,
kills, or changes a phase, and it must not grow one — a script that recommends
standing an agent down is read as having decided, and its phrasing then carries
weight its evidence does not.

Apply the gate and containment checks below and reach your own conclusion.
Destructive steps still follow the standing reap preconditions and pacing rules;
nothing here shortcuts them.

### Standing reap preconditions — ALL must hold

Reaping an agent (removing its worktree) is safe only when every one of these is
true. It is a gate, not a checklist to rationalize past:

1. **Idle > 48h** — no activity in its transcript for more than two days (rank
   by message CONTENT, not mtime; your own broadcasts reset mtime).
2. **No messages exchanged in the last 24h** — nothing sent to it or from it.
3. **Task complete AND nobody is waiting on it** — it finished and its
   dispatcher/manager moved on, so a reply it waits for will never come. (A
   pushed branch held on a still-queued merge is the one nuance — see the
   held-pending-merge rule.)
4. **Salvage done first** — its only-copy state is copied to a durable location
   and verified (see Containment and the reap steps).

**When a needed decision or merge is delayed, ESCALATE — never silently wait.**
If an agent is parked past the 24h window on something its manager/owner has not
acted on, raise it explicitly to whoever owns that decision, and re-raise on
every subsequent run. If the human owner is unresponsive, escalate through your
designated escalation contact (the human's proxy) rather than leaving the agent
parked. A silent park is the failure this whole procedure exists to prevent:
each run must drive a parked agent toward a terminal state — decision made /
merge lands (reap-able) or deemed orphaned (reap).

### The instruments, in order of trust

| Question                    | Instrument                                              |
| --------------------------- | ------------------------------------------------------- |
| Is it alive?                | `thrum tmux connect < /dev/null` · `tmux list-sessions` |
| What did it last say?       | its transcript JSONL — **the primary signal**           |
| How long has it been quiet? | transcript mtime — **weak, see below**                  |
| What is it waiting for?     | its own last message, read as prose                     |

**Do NOT use `thrum team`, `phase`, or `last_seen_at` for any of this.** They
are stale, and they fail toward reporting a live agent as absent.

### Locating an agent's transcript

The project directory is the agent's worktree path with `/` and `.` replaced by
`-`:

```
/Users/<you>/.thrum/worktrees/thrum/<session>
  -> ~/.claude/projects/-Users-<you>--thrum-worktrees-thrum-<session>/*.jsonl
```

Newest `.jsonl` by mtime is the live session. Records carry `timestamp`, `type`,
and `message.content`.

### The scan

```bash
thrum tmux connect < /dev/null | awk 'NR>1 && $1 ~ /^[0-9]+$/ {print $2"|"$3}'
```

gives `session|@agent` for every alive agent. For each, open the newest
transcript and take the last record of `type == "assistant"` with a non-empty
text block. That text is the signal.

### Reading the signal

**Terminal-state language** — the agent declaring it holds nothing:

> _standing by_ · _standing down_ · _task done_ · _idle_ · _no further action_ ·
> _completion report sent_ · _held pending X_ · _inbox clear, nothing to act on_

**Not terminal**: a question to another agent, a partial result, a stated next
step, an error it has not resolved.

### ⚠️ Two false positives the phrase match produces, both measured

The scan is a **pre-filter**. It matches anywhere in the last message, which
catches two agents that are not finished:

1. **A long active report that happens to contain a terminal phrase mid-text.**
   The agent is mid-work; the phrase is describing something else.
2. **A freshly-primed agent.** _"Inbox empty; standing by"_ is part of its
   prime, not a completion. Check its session start — minutes old means it just
   woke.

⇒ **Read the END of the message, not a keyword hit.** A finished agent's last
sentence IS the terminal statement; a working agent's merely contains one.

### 🔴 THE GATE — an agent saying it is finished is not evidence that it is

**Name what it is waiting for, then prove that thing is dead or landed.**

_"Held pending the coordinator merge"_ is finished only if the merge landed or
was abandoned. If the merge is still queued, whether the agent is _correctly_
waiting depends on its branch and its idle time — see the held-pending-merge
rule below.

**If you cannot name the blocker, it is not a candidate.** (This governs the
_no-transcript_ case — absence of evidence — not a named-but-stale merge wait.)

### 🔴 The held-pending-merge case — a PUSHED branch changes the gate

An agent parked "pending the merge" is judged by ONE thing first — **is its
branch pushed?** — because a merge wait only makes sense if the merge king can
reach the branch.

- **Branch NOT pushed — broken at ANY idle age.** The merge king cannot access
  an unpushed branch, so the wait can never resolve, and the agent's commits are
  also the only copy (unsafe to reap). This is not a legitimate wait — it is a
  stuck one. → Get the branch PUSHED (nudge the agent / its orchestrator to push
  it, or escalate that it needs pushing). Never reap an unpushed
  held-pending-merge agent — pushing first is both what unblocks the merge and
  what makes any later reap safe.
- **Branch pushed, idle < 24h.** Legitimately waiting its turn in the merge
  queue. Leave it (the pushed branch means the merge king can reach it and
  nothing is lost).
- **Branch pushed, idle ≥ 24h.** The wait is no longer legitimate — a pushed
  branch is not the only copy of anything, so any agent could carry the merge.
  Resolve it to one of two clear states:
  - **Merge HAPPENED** (branch is an ancestor of trunk, or was
    abandoned/superseded) → the agent is **ORPHANED**, waiting for a response
    that will never come → **reap** (salvage-first, and still subject to the
    standing reap preconditions).
  - **Merge still PENDING** (branch not on trunk, not abandoned) → the merge
    king forgot it or the queue is stuck → **ESCALATE** to the merge king (name
    the branch tip + how long it has waited), and **re-escalate on every
    subsequent skill run** until it resolves. Never leave it silently parked.

**The escalation is the load-bearing half.** Because each skill run re-checks
and re-escalates a still-pending merge, the condition is self-correcting: it
drives toward one of two terminal states — the merge completes (agent then
reap-able) or the agent is deemed orphaned (reap) — instead of a pushed-branch
agent parked forever on a merge nobody is tracking. A silent park is exactly the
failure this rule prevents.

### Containment, before anything is stood down

- branch pushed, or no commits to lose
- no only-copy untracked state:
  `git -C <wt> status --porcelain --untracked-files=all --ignored -- .thrum` —
  **both flags**; the narrow form returns a false zero
- no restart snapshot under `<worktree>/.thrum/restart/` — a snapshot means a
  sleeping agent that expects to wake with its context

### Reaping many worktrees at once — the batch path

The single-agent gate above still governs EACH worktree; batching changes only
how you GATHER the evidence and EXECUTE, never the judgment.

**Gather with read-only examiner sub-agents.** For dozens of candidates,
partition them across background sub-agents that examine each worktree read-only
and RETURN a verdict — they never reap. Per worktree each reports:
sleeping-agent marker present?; branch containment (ancestor of trunk / pushed /
UNMERGED-with-commit-list); only-copy authored content (docs/plans/specs,
undelivered reports, restart snapshots), ignoring regenerable dirt (`thrum-*.sh`
helper scripts, `.thrum/redirect`, `runtime_config.json`). Fence them READ-ONLY
(no checkout/reset/stash/clean/rebase/commit/push, no `worktree remove`, no
`agent delete`, no `rm`). You consolidate and decide.

**Cross-check the reap set against the live + sleeping roster before ANY
deletion** — one protected agent in the set aborts the batch. Build the
exclusion from the live-pid set + sleeping set + locked worktrees, not from
names.

**Salvage only-copy state FIRST, and diff by NAME.** Copy with `cp -a`, then
compare source vs salvage with `find -type f | sort` (set-diff by NAME) BEFORE
trusting per-file `cmp` — a flat copy of two same-basename files (e.g.
`.thrum/restart/<a>.md` and `.thrum/context/<a>.md`) silently clobbers one, and
`cmp` on what copied passes clean. Give salvaged files distinct names and land
the salvage somewhere durable (the main repo, not `/private/tmp`).

### 🔴 The reap is worktree-removal-only — and TEST every step on ONE item first

**`thrum agent delete` refuses an agent whose linked worktree still exists**
(and any agent with a live pid/tmux process, and coordinators); `--force` does
NOT bypass these guards. So the reap that frees load is worktree removal first —
salvage BEFORE you remove, then remove PLAIN (no `--force`), then
`thrum agent delete`:

1. Enumerate untracked+ignored `.thrum` content — BOTH flags, one flag alone
   misses a class: `git status --untracked-files=all --ignored -- .thrum`
2. Salvage BOTH kinds to their canonical redirect paths, and `cmp`-verify each
   copy is byte-identical to its source:
   - `?? .thrum/agents/<agent>/sessions/*-restart.md` — the restart snapshot.
   - `!! .thrum/context/<agent>.md` — gitignored, NOT daemon-held, the **only
     copy**; losing it is permanent. The file is no longer loaded at startup,
     but its existing contents still need salvaging. (`_preamble.md` is
     regenerable — skip it.)
3. THEN `git worktree remove <path>` — **plain, no `--force`.** ⚠️ This refusal
   is a REAL backstop only for untracked/modified content (git's own rc=128
   refusal) — it is NOT a backstop for the gitignored
   `.thrum/context/<agent>.md` file: `git worktree remove` silently deletes
   ignored-only content with exit 0, no refusal, no warning (verified by direct
   reproduction). **Step 2's salvage is the ONLY protection for that file —
   there is no git-level safety net behind it.** The rc=128 refusal still
   matters (it catches a stray untracked restart-snapshot salvage missed, or any
   other unexpected untracked file), so still read the BARE rc and still treat a
   refusal as a signal to investigate — just don't mistake it for protecting the
   ignored-file class. (Not a pipe — a pipe returns the last command's rc, not
   `git`'s.)

Then REAP the agent row with `thrum agent delete <agent>` once the worktree is
gone and no process is live: orphan rows must be reaped, not left. Never use
`thrum agent cleanup --force` (it can flag live fleet agents). A refused reap on
a dead, worktree-less agent is a bug: report it, do not route around it.

⚠️ **After a large reap you may see a `DeadAgentSweeper` skip alert + elevated
write-RPC latency — do NOT assume the reap caused it.** This box carries a
standing single-writer contention condition (one serialized write conn,
`SetMaxOpenConns(1)`) that spikes write-RPC latency recurrently and
**reap-independently**. A large reap is at most ONE coincident spike: stranding
dozens of stale agents (worktreeless, empty `pid_start_time` → always "due")
stretches the sweep's evidence-gathering run-length (that is what `held=`
measures — a run duration, mostly off-lock syscalls, not a lock hold or a
deadlock; the skip is correct, the guard releases via `defer`), but it drives
almost no writes of its own. The discriminator that it is contention and not
poison: `write_pool in_use=0`, `selfheal_write_poison_detected=0`. **So do not
chase the reap as the lever** — pacing/batching large reaps is mild hygiene at
best (it trims run-length and a little pressure), a restart is transient relief
only, and the durable fix is **writer-decontention**, owned by the daemon-perf
agent. Symptom to recognize (reads and `daemon status` stay ~0.5s while
`agent status` climbs — writes slow, not reads): hand it to the daemon-perf
owner, don't re-diagnose the internals yourself.

**Before running any batched destructive loop, run it on ONE item and verify the
EFFECT** (agent gone from the registry / worktree gone from `git worktree list`
and disk), not just the exit code. A batch can fail wholesale for a reason a
single test surfaces instantly — a minimal-`PATH` loop/background context where
`git`/`thrum`/`sleep` are `command not found` (set `PATH` explicitly or use
absolute paths); a `read` that collapses a leading empty field. Batched worktree
removal is also SLOW on a loaded box (seconds each) — run it in the background
or in chunks, never one 2-minute foreground loop.

### ⚠️ Idle time is corrupted by your own traffic

A coordinator broadcast resets every recipient's transcript mtime. After any
fleet-wide message, **every agent reads as freshly active** and the idle column
is meaningless for hours.

⇒ **Rank by the last message's CONTENT, not by elapsed time.** A
nudged-then-idle agent re-states that it is standing by, so the content signal
survives what the timing signal does not.

### ⚠️ Two absences that are not evidence

- **No transcript** does not mean no agent. Alive agents exist with no
  transcript directory.
- **Absent from `connect`** means only "no tmux session on THIS box". `connect`
  is box-local; remote agents can never appear in it.

### Idle-gopls RAM reclaim

Once you've classified an agent as idle, its `gopls` process (if any) is a
second, independent reclaim candidate — an idle agent's gopls is dead weight
sitting in RAM.

Match each running `gopls` process to its owning agent (ppid chain → tmux
pane_pid → `thrum team --json` session) and use `tmux_state` as a cheap
first-pass filter — anything not idle by that signal is skipped outright. That
filter is bounded, not authoritative: it does not replace this skill's own
idle/finished judgment, which still comes from the transcript per the rules
above. Do this alongside the scan above, apply the same transcript-based gate to
whatever it surfaces, and fold the result into a single combined table: idle
agent · gopls PID · RSS-MB.

- Killing an idle agent's gopls is a no-op — it respawns on the agent's next LSP
  request. Never destructive to the agent itself.
- **NEVER kill a pool or active-worker gopls.** Exclude by ROLE (orchestrator,
  gate, coordinator, brainstormer, researcher — the roles that are never reap
  targets) plus the non-idle-state skip above. Before building any kill-list,
  prove the exclusion still resolves correctly; if you cannot, abort rather than
  print one. Do not override or bypass any of this.
- Same framing as the rest of this skill: this step **produces information, you
  decide**. It is read-only — it yields a candidate list, it never kills
  anything.

### 🔴 Footguns — never do these (each measured, each cost real time or state)

- **Never glob/`cp` `.thrum/context/*.md` when salvaging** — the glob matches
  the shared `project_state.md` and overwrites the canonical copy with a stale
  worktree copy. Salvage the agent-specific `<agent>.md` by exact name only.
- **A worktree's `.thrum/` holds a redirect file to the main repo AND a physical
  stale local copy of shared files** (e.g. `project_state.md`) — the physical
  worktree copy is NOT canonical; never treat it as the source of truth or copy
  it upward.
- **Never use shell-specific builtins in a reap loop without checking the
  shell** — `mapfile`/`readarray` are bash-only and silently fail in zsh
  (`command not found`), so the salvage you thought ran did not. Use portable
  `while read` and verify each salvaged file actually landed.
- **`rm -rf` raises a confirmation modal that blocks the PARENT pane invisibly**
  — use `rm -r` for scratch cleanup (identical deletion, no modal). Never
  `rm -rf`.
- **`git worktree remove` (plain) silently deletes gitignored-only content
  (rc=0)** and refuses (rc=128) only on untracked/modified files — salvage is
  the ONLY protection for gitignored only-copy state.
- **`git worktree remove` refuses while a just-salvaged untracked snapshot still
  sits in the worktree** — remove that salvaged copy from the worktree first,
  then plain-remove.
- **Never `checkout`/`reset`/`restore`/`stash`/`clean` in a shared tree, even to
  undo** — to restore a file to its committed version, write it with a plain
  redirect from `git show <ref>:<path>`, not a checkout.
- **Test the reap on ONE agent first and verify the EFFECT** (gone from the
  worktree list AND disk), not just the exit code.
- **Rank candidates by last-message CONTENT, not elapsed time** — your own
  broadcasts reset every recipient's idle clock.
- **`--force` on agent-delete does NOT bypass its guards** — the worktree must
  go first (teardown with salvage or plain `git worktree remove`), then
  `thrum agent delete` reaps the row.
- **Reap order is `thrum tmux kill <session>` FIRST, THEN
  `git worktree remove`** — removing the worktree alone leaves the agent's tmux
  session alive with a now-deleted cwd, which the fleet flags as recurring "cwd
  drift" and whose runtime process keeps consuming RAM/CPU. Kill the session,
  verify it's gone, then remove the worktree.

### Why this fails dangerously if you skip the gate

The failure direction is **false-GONE**: concluding a live agent is finished. It
authorises abandonment, and **nobody audits an agent nobody believes exists**,
so the error has no natural discovery path. A false-ALIVE costs one wasted
check. Prefer it.