Skill Stocktake

---
name: skill-stocktake
description: Audit installed Claude skills for quality and surface Keep/Improve/Update/Retire/Merge verdicts. Use when the user says "audit my skills", "stocktake", "review my skills", "which skills should I retire or merge", "do a quality pass over my skills", or "/skill-stocktake". NOT for creating or improving a single skill (that is skill-creator).
license: MIT
metadata:
  author: shimo4228
  version: "3.2"
  extracted: "2026-02-21"
origin: shimo4228
---

# skill-stocktake — Skill Quality Audit

Evaluate installed skills and assign each a verdict: `Keep / Improve / Update / Retire /
Merge`, plus `Out of scope` for skills this harness does not own (Phase 0). This skill
does NOT do the improving — once it has a verdict, it **hands off to skill-creator (the
improvement engine)**. That boundary is the point: stocktake is the quality gate,
skill-creator is the fixer.

> Design note. Per-item quality/freshness and set-level existence/overlap are
> different properties. Deterministic checks are code-owned and unconditional;
> per-item judgment gets narrow, fresh contexts; set-level judgment gets a light
> description sweep with targeted deep reads.

## Modes (`$ARGUMENTS`)

| Argument | Behavior |
|----------|----------|
| none / `full` | Evaluate every skill (default) |
| `changed` | Re-evaluate only skills whose `SKILL.md` mtime is newer than `results.json`'s `evaluated_at`; carry the rest forward from the ledger. Phase 0 and Phase 3 still run over the full set (structural debt, overlap, and contradiction are set properties) |

`changed` detects changes inline (no script):
```bash
find ~/.claude/skills -name SKILL.md -newermt "$(jq -r .evaluated_at ~/.claude/skills/skill-stocktake/results.json)"
```

## Phase 0 — Structural pre-pass (deterministic, code-owned)

Run BEFORE any LLM judgment. These checks never ride on per-item attention:

1. **skill-health scan** — dangling references / missing artifacts across the library
   (run its scanner per `~/.claude/skills/skill-health/SKILL.md`). Mandatory, not
   optional: this is the structural layer of Curate — code enumerates the debt,
   judgment decides — and judgment-owned checks let exactly these defects through.
2. **Ledger hygiene** — for every `results.json` entry: the path must exist on disk and
   the key must follow the canonical rule below. Dedup violations (same path under two
   keys → keep the canonical key, delete the duplicate). Entries whose path no longer
   exists are removed (note them in the report as retired-from-disk).
3. **Existence before judgment** — a skill that fails the existence check never reaches
   Phase 2. Do not let an LLM assign Keep to a file that is not there.
4. **Ownership before judgment** — the same skill-health run reports `external`: skills
   whose directory is a symlink out of the skills root (a package manager's tree, a
   sibling checkout). **These do not get an Improve/Update verdict.** Record them as
   `Out of scope` with the owner and real path in the reason, and route any real defect
   **upstream** (issue / PR). A verdict here is not actionable — an edit lands in
   someone else's tree, never reaches git, and disappears on their next upgrade
   (regression: [modem-dev/hunk#595](https://github.com/modem-dev/hunk/issues/595)).
   The check is fully structural (`is_symlink()`), so it belongs in code, never in
   per-item attention.

## Phase 1 — Inventory

Enumerate skill definition files with Glob (no script needed):

- `~/.claude/skills/*/SKILL.md`
- if cwd has `.claude/skills/`, also `{cwd}/.claude/skills/*/SKILL.md` (project skills)

**Canonical ledger keys**: one key per skill directory name, matching the path on disk
(ADR-0047). A rule or skill referencing a name in prose does NOT make it a skill.

**Usage evidence** (parent-owned; batch agents never see it). Step 0 — run the script and
transcribe the numbers; do not re-derive them by hand:

```bash
uv run --frozen --project ~/.claude/skills/skill-stocktake python -m scripts.usage_stats --days 14
```

JSON out, exit 0 always: `counts` (per skill — `deliberate` / `slash` / `invoke` /
`last_used`, ordered by deliberate use), `window_label`, `measurable`, and `excluded`
(how many rows each correction removed). The four corrections that decide whether the
number means anything — the event-type split, the two sandbox exclusions, and the
real-span label — live in the script's docstring together with their measured effect on
the real log. **Do not re-implement them as a jq one-liner here** (ADR-0052): the
`verify-bootstrap` case is why — 2 apparent uses, 0 after correction.

Two readings the script hands you but cannot make for you:

- `measurable: false` → render usage as `—` (unmeasured). **Never render it as 0** —
  unmeasured and unused are different facts.
- `span_shorter_than_window: true` → label the column with `window_label` (the log's real
  span), not the nominal `14d`.

**Reference-URL evidence** (parent-owned for the same reason, plus one of its own: Phase 2
runs batch agents *in parallel*, and `fetch named URLs` inside each of them is a burst
against the same hosts — the shape `rules/common/debugging.md` forbids). Check every URL
the library names once, serially, here:

```bash
set -o pipefail   # without it a producer crash reads as "0 dead links, all healthy"
uv run --frozen --project ~/.claude/skills/skill-health python -m scripts.scan_refs --external-urls |
  uv run --frozen --project ~/.claude/skills/skill-health python -m scripts.url_liveness --urls-from -
```

Extraction is `scan_refs`'s job, not a `grep`: it skips fenced code blocks and template
slots, so the audit does not fetch its own example URLs. (`--offline` on `url_liveness`
is the dry run — it reports every URL `skip` without touching the network.)

Verdicts are `live` / `dead` / `blocked` / `skip`. **`blocked` is not `dead`** — a 403 is
bot policy, not absence; reporting it as dead sends the next reader chasing a link that
is fine. `skip`, `offline_suspected: true`, `halted: true`, and a non-null `source_error` all
mean *unchecked*, not *fine* — and `input_lines: 0` means nothing was ever checked, which
is not the same as a clean corpus. If the run halted on a rate limit, say so in the report and do **not** re-run the
remainder in this session.

**Residency-cost evidence** (parent-owned, once per audit). The substrate's own
report is the only instrument that measures what a skill's *listing line* costs every
turn — the number RFC-0017 / RFC-0018 argue about. Run it from the parent, once,
from the repo cwd when project skills are in scope:

```bash
claude -p "/skill-doctor"
```

The child inherits the user's settings, and that inheritance is the measurement: the
listing it reports is the one real sessions carry. Run it without `--settings`
isolation. It prints a text table (no JSON; Claude Code ≥ 2.1.269, as-of 2026-09-15 —
the report is substrate-owned, so when its printed legend and this section disagree,
the legend wins). Transcribe per skill:

- `context` — tokens the skill's listing line adds to the system prompt every turn.
  `-` means not in the listing (`disable-model-invocation: true`, or not loaded) and
  costs nothing. This is the residency cost; nothing else in the harness measures it.
- `uses` / `last used` — the substrate's count, with its window unlabelled (rows whose
  `last used` is far outside 7 days still carry counts, so read it as cumulative). It
  includes the sandbox child sessions `usage_stats` corrects out, so it is a
  **cross-check** for the deliberate-use column, never its replacement. Where the two
  disagree beyond what the sandbox correction explains, say so in the report; do not
  average them.
- `7d tokens` — tokens attributed to sessions that loaded the skill's body over the
  last 7 days. Reference only.
- The "loaded but never invoked" list — a cross-check against Phase 4's
  description-audit candidates, which come from Phase 2 quotes joined with
  `usage_stats` deliberate use. A skill on this list but not among the candidates
  (or the reverse) is a disagreement to report, not a candidate to add.

`Skill usage reports are not available on this connection.` means feature-flag
fetching is off (`DISABLE_TELEMETRY`, `DO_NOT_TRACK`,
`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`, or a third-party provider): render
`context` as `—` (unmeasured) and say so up front.

State the scan result up front: which paths were scanned, how many skills found,
whether usage is measurable, and whether residency cost was measured.

## Phase 2 — Per-item scrutiny (parallel small batches, fresh contexts)

Split the target set into batches of **10–12 files** and launch **one subagent per batch in parallel**. Small batches are
the point: per-item attention dilutes as a context fills. Do NOT pass prior verdicts or
the ledger to batch agents (anchoring); do NOT pass usage or residency-cost data
(parent-owned dimensions).

Each batch agent applies, per skill:

**Stage 1 — binary screen.** Explicit Yes/No per item; surface only the No answers
(these six questions are the canonical set — `skill-creator` §4 reuses them for its
creation-time draft gate by reference, not by copy):

- [ ] Actionability: concrete steps/commands/examples you can act on?
- [ ] Scope fit: name, trigger (description), and body aligned — not too broad or narrow?
- [ ] Within-batch uniqueness: no content overlap with other skills in this batch?
  (a documented orchestrator/sub-skill split is NOT overlap; cross-batch overlap is
  Phase 3's job, not this agent's — the same goes for contradictions with skills outside
  this batch)
- [ ] Currency: **unconditionally verify** every artifact the skill names — `ls` each
  referenced path (`~/.claude/agents/<name>.md`, hooks, bundled scripts) and run
  `--help` / version checks for named CLI flags. "Verify if it looks stale" is banned
  phrasing: the condition is what dilutes. Deterministically checkable claims get
  deterministic checks, every time. **Do not fetch URLs** — the parent checked them once
  in Phase 1 and hands you the verdicts; parallel batch agents each fetching is a burst.
  Read the skill's `MAINTENANCE.md` when it exists: it holds the reasons and history the
  body leaves out, and a claim there can be as stale as one in the body.
- [ ] Hygiene: is the body free of bloat — trivial prohibition lists, repeated emphasis,
  and step-by-step recitals that could fold into a principle (the skill-creator §3 writing guidance)?
  Do NOT fold greppable detection terms, self-enforcing prohibitions, or numeric
  thresholds — abstraction destroys their function (ADR-0058 rejected uniform
  shortening for exactly this). Bodies grow after creation; this question is the
  standing check the creation-time gate cannot repeat. Also: does the body run on its own —
  no value or rule needed to act is left to a file outside the skill directory (an ADR, a
  rule section)? A parenthetical ADR number that only names *why* is provenance, not
  delegation; it belongs in the skill's `MAINTENANCE.md`, which stays unpublished.
- [ ] Description audit: is the description free of standing instruction text — NOT-for
  routing, "Use PROACTIVELY"-style fire directives, mentions of other skills — beyond
  stating what the skill is for and when to reach for it? A listed description resides
  in the system prompt every session as an unaudited instruction layer (RFC-0018).
  Quote each instruction sentence found (semantic judgment, not regex); do NOT judge
  its effectiveness — the parent joins quotes with usage and residency cost at
  synthesis.

**Stocktake-only existence pass (every item, including Keep-bound skills).** This is
separate from the canonical six-question quality screen above, which `skill-creator`
reuses. Answer both questions explicitly:

- [ ] Standalone value: if this file disappeared, would the library lose a user job that
  is not already covered by an installed skill, rule, runtime substrate, or cheap
  on-demand generation?
- [ ] Cost asymmetry: does the skill's independent trigger and reusable judgment justify
  its selection, drift, and maintenance cost?

A No answer makes the skill a non-Keep candidate and must be pressure-tested. Do not
shield a skill from this pass merely because it is current, well written, recently used,
or part of a documented layer.

**Stage 2 — verdict pressure-test (non-Keep candidates only).** Generate 1–3
skill-specific atomic yes/no questions that try to **refute the draft verdict** before
finalizing it. Answer each with one line of evidence (file read, path check, WebSearch).
A refuted defect → fall back toward Keep. A confirmed defect → the No answers become
the concrete improvement list handed to skill-creator (Improve/Update) or the removal
rationale (Retire).

Evaluation is **holistic judgment, not a numeric rubric** — binary answers are evidence
feeding the verdict, never aggregated into a score (a satisfaction ratio changes no
decision and dilutes a single dominant No). Batch agents return structured verdicts
with self-contained reasons. Every skill gets the fixed existence pass; only non-Keep
candidates get the dynamic pressure-test questions.

## Phase 3 — Overlap and contradiction probe (one dedicated agent, set-level)

Duplication **and contradiction** are set properties; they get a specialist, not a side
effect. A per-item batch cannot see either — each agent reads only its own 10 files and
finds nothing wrong with any of them individually. One agent, whole library:

1. Read **name + description (+ heading structure)** of every target file — light pass.
2. Propose candidate clusters greedily (false positives are fine at this stage).
3. For each candidate cluster, read the bodies side by side and judge:
   - `ABSORBABLE_OVERLAP` — two skills serve the same user intent, trigger, and job;
     maintaining both adds selection/drift cost; the source has named unique residue
     that can move into a named Merge target
   - `FULLY_ABSORBED` — another asset already covers the whole user job and the source
     has no unique residue to move; name the covering asset and use Retire, not Merge
   - `CONTRADICTION` — two skills that can both load give **opposite instructions for the
     same situation**, or a sub-skill's content violates the canon it defers to. Quote
     both sides. See the contradiction checks below.
   - `DOCUMENTED_LAYERING` — orchestrator→sub-skill, rule→skill, declared defer; quote
     the defer line and name each file's independently triggerable responsibility
   - `ADJACENT_BUT_DISTINCT` — near domain, different job; name one concrete independent
     user request that each member handles
4. One extra pass over `~/.claude/rules/` and MEMORY.md: is a rule re-stating a skill
   (or vice versa) beyond a declared pointer? Flag promotion residue (a skill whose
   content a rule has fully absorbed) as Retire/Merge candidates.

### Contradiction checks (a declared boundary is a claim, not evidence)

`DOCUMENTED_LAYERING` certifies "not duplicated". It does **not** certify "not
contradictory", and a quoted defer line is the start of the check, not the end. For every
cluster labelled `DOCUMENTED_LAYERING`, run all three:

- **Does the subordinate file comply with the canon it defers to?** Read the canonical
  file's rules and look for the subordinate stating a *different* rule for the same
  decision — especially **defaults** ("if the user gives no X, use Y"), which contradict
  quietly because they only fire in the unspecified case.
- **Is the defer bidirectional?** A one-directional defer (canon claims the subordinate,
  subordinate says nothing) means the subordinate **loads alone without its correction**
  whenever its own description wins the trigger. A broad description on a subordinate
  file turns a latent contradiction into an active one.
- **Do the two files presuppose the same author / project model?** Imported skills
  (`origin` = an external repo) carry their author's premises. A conflict of premises
  reads as a normal-looking instruction and survives every per-file check.

`ABSORBABLE_OVERLAP` produces Merge verdicts because named residue must move into the
target. `FULLY_ABSORBED` produces Retire because the target already covers the whole job
and there is nothing to move. `CONTRADICTION` produces Improve (add the missing defer,
delete the conflicting rule, narrow the trigger) or, when the premises themselves clash
and the subordinate never fires on its own, Retire-and-absorb — name what must move
before deletion.

## Phase 4 — Synthesis (parent)

Merge Phase 2 verdicts, Phase 3 overlap / contradiction verdicts, and parent-owned usage data:

- **Content owns the verdict; usage is reference evidence only.** Render the 14-day
  deliberate-use count and last-used date; usage is never a prerequisite, veto,
  threshold, or automatic candidate trigger.
- **Retire content-first.** A skill can be Retire regardless of usage when (a) its
  workflow or underlying premise is obsolete, (b) another asset actually absorbs its
  user job and no standalone residue remains, or (c) it is too thin, non-actionable, or
  cheaply regenerated for its selection/drift/maintenance cost. State which route the
  evidence supports and what covers the need instead.
- **Separate existence from freshness.** A stale version pin, retired CLI flag, or dead
  pointer that is locally fixable is Update, not Retire. It becomes Retire only when the
  verified evidence shows the workflow/premise itself is no longer worth preserving.
- **Charge aggregate cost through the existence pass.** Holding a skill adds selection,
  drift, and maintenance cost; do not protect a merely adequate skill because it has no
  individual defect. `disable-model-invocation: true` (`context` = `-`) and a name in
  another skill's chain row are **not** grounds for a Yes on cost asymmetry: the first
  prices selection cost alone and leaves drift and maintenance cost unpaid, the second is
  a pointer, which is a claim and not evidence of use. For a reference-style skill with
  zero invokes, pressure-test existence with its **organic reads** — read events in
  `~/.claude/metrics/skill-usage.jsonl` with audit days excluded (this audit's
  `evaluated_at` and any generation-audit / sub-audit dates), since a stocktake reading its
  own targets registers as reads. A few organic reads that stop months back is evidence the
  file is no longer functioning as a reference (ADR-0068).
- **Description-audit quotes join usage and residency cost here.** An instruction
  sentence whose aim (self-firing, misfire routing) the deliberate-use record does not
  support is a removal candidate: fold it into the body or an existing rule. The
  `context` column prices the candidate — a candidate with nonzero `context` and zero
  deliberate use is the RFC-0017 fold case (the fold recipe is skill-health Phase 3, Utility), and a
  skill already at `-` has nothing left to fold. A
  supported trigger surface is left alone (do not strip descriptions that demonstrably
  fire). Residency cost is a price, not a verdict input: it orders the fold candidates,
  it never makes one.
- Conflicts (e.g. batch says Keep, probe says Merge) are resolved by the parent reading
  the cited evidence, not by vote.

| Verdict | Meaning |
|---------|---------|
| Keep | Useful, current, unique value |
| Improve | Worth keeping, but specific improvements needed |
| Update | Referenced technology/artifact is outdated (verified, with evidence) |
| Retire | Obsolete, cost-asymmetric, or `FULLY_ABSORBED` with no residue to move |
| Merge into [X] | `ABSORBABLE_OVERLAP` with named unique residue; name the target and residue to move |
| Retire-and-absorb | A `CONTRADICTION` whose premises clash and whose subordinate never fires on its own; name what must move into the canon before deletion |

Evaluation is **origin-blind**: the same checklist applies to every skill.

Render a table: `Skill | ctx | 14d | last used | Verdict | Reason`, where `ctx` is the
`/skill-doctor` `context` figure (`—` when unmeasured) and the count is **deliberate
use only** (`slash + invoke`). If the log is younger than 14 days, replace `14d` with
its real span.

## Phase 5 — Consolidation

**Confirm one by one**: walk the non-Keep candidates
sequentially — for each, show the evidence first, then ask `[y/n/skip]`. Never batch the
approval. The user can stop at any point; `skip` records the verdict in the ledger
unactioned.

- **Retire / Merge**: per file, present (1) the specific defect found, (2) what covers
  the same need instead, (3) the impact of removal (dependent skills, rule pointers,
  MEMORY references) → ask `[y/n/skip]`. **Act only after the user confirms that file.**
- **Improve / Update**: **offer** per skill — "Hand `<skill>` to skill-creator?
  `[y/n/skip]`" — and on approval invoke `skill-creator` with the target skill and the
  No-answer list. Stocktake never does the improvement work itself.
- **Update the ledger**: Read `results.json` → merge this run's verdicts → Write it back
  (`evaluated_at` = real UTC from `date -u +%Y-%m-%dT%H:%M:%SZ`). In `changed` mode,
  preserve the prior verdicts of skills you did not re-evaluate.

## Reason quality (required)

Every `reason` must be **self-contained** — decision-enabling on its own. "unchanged"
alone is banned; always restate the evidence. For non-Keep verdicts, the reason cites
the **No answers from the binary screen / pressure-test** (question + one-line evidence).

- **Retire**: state the defect + the replacement. Bad: `"Superseded"` / Good:
  `"disable-model-invocation: true already set; continuous-learning-v2 covers the same
  patterns plus confidence scoring. No unique content remains."`
- **Merge**: name the target + what to integrate. Bad: `"Overlaps with X"` / Good:
  `"42-line thin content; Step 4 of chatlog-to-article already covers this workflow.
  Integrate the 'article angle' tip there as a note."`
- **Improve**: which section, what change. Bad: `"Too long"` / Good: `"276 lines;
  'Framework Comparison' (L80–140) duplicates ai-era-architecture-principles. Delete it
  to reach ~150 lines."`
- **Keep** (mtime-only change in `changed` mode): restate the original rationale.

## results.json (lean ledger)

```json
{
  "evaluated_at": "2026-06-24T11:09:00Z",
  "skills": {
    "<skill-name>": {
      "path": "~/.claude/skills/<name>/SKILL.md",
      "verdict": "Keep",
      "reason": "...",
      "mtime": "2026-01-15T08:30:00Z"
    }
  }
}
```

A ledger for verdict history and the last-audit timestamp only. Update it inline with
Read/Write, not a script. Global skills only (project skills are read fresh from cwd,
not cached here). Phase 0's hygiene rules (canonical keys, existing paths) are part of
every write.

## Related

- `skill-creator` — the improvement engine; hand off Improve/Update work to it.
- `skill-health` — the deterministic structural layer (dangling references, ownership,
  the description-fold recipe).
- `rules-stocktake` — the same audit for `~/.claude/rules/` (residency cost instead of
  usage).
- `agent-stocktake` — the third sibling, for `~/.claude/agents/` (hybrid cost model:
  description = residency, body = invocation).
- `generation-audit` — on a model-generation change, collects runtime-layer evidence
  (conflict / redundancy) and `/claude-api prompt-audit` findings, and hands the skills slice to Phase 4 synthesis as
  external evidence (read, never require).
- `harness-boundary` — design-time lens (layer / portability / obsolescence) for proposed
  mechanisms; applied to an installed skill, its Delete / Move are Phase 4 evidence only.
- `repo-asset-stocktake` — the same stocktake pattern for a project repo's non-code
  assets.
- `llm-as-judge` — the generic judge design canon (binary screen → pressure-test →
  holistic named verdict, no aggregation); Phase 2 is its library-scale implementation.
- `harness-sync` — use it to sync this skill to its public repo.

## References

The two-stage binary-question design (screen → verdict pressure-test, holistic verdict,
no score aggregation) follows the checklist-decomposition evaluation line: BinEval
"Ask, Don't Judge" ([arXiv:2606.27226](https://arxiv.org/abs/2606.27226)), CheckEval
(arXiv:2403.18771), TICK (arXiv:2410.03608). Scores are deliberately not adopted:
BinEval's own limitations show over-decomposition degrades correlation on holistic
quality dimensions, and a satisfaction ratio would dilute a single dominant No.

The hybrid architecture separates deterministic verification, narrow per-item
scrutiny, and set-level existence/overlap judgment. The split preserves attention for
each property without carrying prior verdict outcomes into the next audit.