Skill Improver

---
name: skill-improver
description: >-
  Autoresearch loop for Claude Code skills — greedy keep/discard hill climbing
  on a 10-dimension quality rubric, with blind subagent validation for
  self-scoring bias, plus a `freshen` mode that probes external references
  (release notes, docs, deprecation signals) and applies verified updates,
  plus a `trigger` mode that measures and tunes the skill's frontmatter
  description until it reliably fires when it should and stays silent when
  it shouldn't (60/40 train/test split, 3 runs/query, blinded test scores).
when_to_use: >-
  Triggers on "improve a skill", "optimize a SKILL.md", "make my skill better",
  "run skill autoresearch", "self-improve skills", "evaluate skill quality",
  "score my skill", "audit a skill", "rate my skill", "refine skill
  description", "iterate on a skill", "freshen skill", "freshen skills",
  "update skill references", "check skill staleness", "is my skill out of
  date", "refresh skill sources", "skill not triggering", "skill didn't
  fire", "skill won't trigger", "skill not invoked", "tune skill
  description", "fix skill triggers", "skill under-triggers",
  "skill over-triggers", "false-positive skill", "make skill trigger",
  "Claude isn't using my skill", or mentions autonomous skill improvement,
  skill quality scoring, skill optimization loops, stale skill content,
  or skill activation problems.
argument-hint: '[improve|score|freshen|trigger|philosophy|batch] [<skill-name>|--all|<glob>]'
---

# Skill Improver — Autoresearch for SKILL.md

> **Core Philosophy:** The human programs the researcher, not the research.
> Apply Karpathy's autoresearch methodology — greedy hill climbing with
> keep/discard against a scalar metric — to autonomously improve Claude Code
> skills.

## Invocation

Argument grammar:

```
/skill-improver <mode> <target> [--opts]
```

- `<mode>` — `improve` (default) | `score` | `freshen` | `trigger` | `philosophy` | `batch`
- `<target>` — skill name (e.g. `gh-cli`), absolute SKILL.md path, `--all`, or glob (e.g. `vllm-*`)
- `[--opts]` — mode-specific flags (e.g. `--iterations 15`, `--probe-budget 30`, `--runs-per-query 5`)

Examples:

```
/skill-improver freshen autoresearch
/skill-improver score gh-cli
/skill-improver improve ~/.claude/skills/helm
/skill-improver trigger vllm-caching
/skill-improver trigger gh-cli --missed "find issue with label X"
/skill-improver batch freshen --all
/skill-improver freshen --group 'vllm-*'
```

If `<mode>` is omitted, default to `improve`. If `<target>` is omitted and mode is not `batch`, prompt the user. For `batch`, the target after `batch` selects the sub-mode (`freshen`, `improve`, or `trigger`, default `improve`); the target list comes from `scripts/scan-skills.sh`. The `--missed "<phrase>"` flag (trigger mode only, repeatable) seeds the eval set with user-reported failures as gold should-trigger queries.

## The Improvement Loop

### Phase 0: Setup

1. Identify the target skill. Accept a path, or run `scripts/scan-skills.sh` (or Glob pattern `**/SKILL.md` under `~/.claude/skills/` and `.claude/skills/`) to list candidates. Do NOT search `~/.claude/plugins/` — those are managed externally.
2. Read the target skill's entire directory: SKILL.md and any references/, examples/, scripts/, assets/ present.
3. **Read `<skill>/references/improvement-backlog.md` if it exists.** This file carries open issues from prior skill-improver runs — ceiling-hit items that require multi-file restructure or author judgment. Do NOT re-propose items already listed there unless new evidence (e.g. the ceiling is now breakable in one iteration due to earlier structural changes). Items resolved mid-loop get moved to the backlog's "Resolved this pass" section in Phase 6.
4. Read **both** `references/quality-rubric.md` (scoring criteria) **and** `references/improvement-patterns.md` (concrete before/after patterns by dimension) from the skill-improver directory. Both are non-optional — skipping the patterns file means later iterations propose changes that miss documented techniques (Pattern 8.2 terminology standardisation, Pattern 6.1 redundancy removal, Pattern 9.3 frontmatter fields). If you find yourself unsure what to try next at any phase, that is a symptom of skipping this read. **Apply the Boris Alignment Check** (rubric §"Boris Alignment Check") on the baseline — three diagnostic patterns (strict workflow scaffolding, up-front context dumps, model-version compensation) cap Dims 6, 4, and 9 respectively. Caps surface as cross-cutting structural issues that should be lifted ahead of cosmetic dim improvements of the same magnitude.
5. Establish a baseline score by evaluating the skill against the rubric.
6. Spawn a blind scoring agent on the baseline (see "Blind Validation" section). First snapshot the skill: `cp -a <skill-dir> /tmp/<skill-name>-baseline`. Then run the agent in the background while the loop proceeds. **This is non-optional.** The baseline blind agent is the only check on Phase 1's self-score — without it the entire run rests on whatever bias the loop's self-scoring carries. If the runtime cannot spawn agents, run the same prompt manually in a fresh session and paste back the result before entering Phase 2. Do NOT proceed to Phase 6 without both a baseline AND a final blind score on record.
7. Initialize a results log (in-memory or scratch file) with header: `iteration | score | delta | status | description`.
8. Log iteration 0 as `baseline`.

### Phase 1: Evaluate (Score the Skill)

Score the skill on 10 dimensions (each 0–10, summed to 0–100) using the detailed criteria and scoring template in `references/quality-rubric.md` (loaded in Phase 0).

**Cold-score discipline.** When scoring at any phase, read the current file fresh and assign each dimension against the rubric criteria with no reference to prior iteration scores. Do NOT compute the new score by adding deltas to the old. Delta math hides regressions in dimensions you weren't watching.

### Phase 2: Hypothesize (Pick One Improvement)

Identify the **single lowest-scoring dimension** (or highest-impact if tied). If the
baseline blind agent has returned with flagged dimensions (2+ gap), use the agent's
specific justification text — not just the number — to inform the hypothesis.
Formulate one specific change:

- What to change and why
- Expected score impact
- Complexity cost (lines added/removed, new files)

Consult `references/improvement-patterns.md` for concrete before/after patterns organized by dimension.

**The simplicity criterion (from autoresearch):** A small improvement that adds ugly complexity is not worth it. Removing something and getting equal or better results is a great outcome. A +1 score that adds 20 lines of noise? Skip. A +1 from deleting redundant content? Keep.

### Phase 3: Mutate (Make the Change)

1. Apply exactly one change to the skill.
2. Keep the diff minimal and focused.
3. Do NOT bundle multiple improvements — one change per iteration so cause is attributable.

### Phase 4: Re-evaluate (Score Again)

1. Re-score the skill using the same rubric.
2. Compare to previous best score.

**Decision rule:**
- **Score improved** → KEEP. Log as `keep`. This is the new baseline.
  **Anomaly gate (+5 or more):** A single change that lifts the total by +5
  or more is presumed inflated until proven otherwise. Do NOT rationalize the
  deltas. Instead: open the rubric fresh, read the current file as if it were
  new, and score each dimension cold. If the cold total differs from the
  delta-math total by 2 or more in either direction, the cold score wins.
  Most +5 jumps shrink to +3 under cold rescore — that is the finding, not a
  failure of the change. Log both totals in the iteration row.
- **Score equal, but simpler** → KEEP. Log as `keep (simplification)`.
- **Score equal or worse** → DISCARD. Revert via `git checkout -- <file>` (or undo the edit if not git-tracked). Log as `discard`.
- **Change broke something** → REVERT. Log as `crash`. Fix and continue.

### Phase 5: Log and Loop

1. Append result to the log: `iteration | score | delta | status | description`. Use a single declared score column for trend math — pick `self` OR `blind` and stay with it across iterations. Do NOT mix self-scores and blind-scores in the same delta column to make iterations look bigger; if both are tracked, log them as separate columns side by side and compute deltas within each column.
2. Print a one-line status, e.g.: `[iter 3] score: 74 (+2) — keep — moved API docs to references/api.md`.
3. Go to Phase 2 and pick the next improvement.

**Reflect (every 5 iterations):** Categorize all iterations by type (simplification,
style fix, restructuring, content addition, trigger tuning). If the last 5 were all
the same category, force the next hypothesis to be a different category. Print:
`[reflect] N kept from <category>, pivoting to <new category>`

**Stop conditions:**
- Score reaches 90+ AND no dimension is below 7.
- **Ceiling mapped:** 5+ consecutive discards spanning at least 2 different
  improvement categories. This is not failure — it means the skill is near its
  quality ceiling. Report as a positive finding: which categories were tried,
  what the ceiling is, and what would require the author's input to break through.
- **Structural ceiling claim requires evidence.** "Structural ceiling" stops
  require at least 2 logged discards naming the patterns that were attempted
  and why each failed. A run with zero discards has not mapped any ceiling —
  it has stopped early. If you find yourself reasoning "the next iteration
  would just be a discard" without actually trying it, that reasoning is the
  cheat. Try it.
- User interrupts.
- 10 iterations completed (default cap; user can override).

**What a stop is NOT:**
- Not "+N feels like enough". The metric drives the loop; subjective comfort
  with the gain does not.
- Not "the score is good and I am tired". Read on.
- Not "Dim X is capped, so further improvement is impossible". Other dims
  may still be liftable. Stop only when the rubric criteria for stopping match.

**On stop:** Spawn a final blind scoring agent (see "Blind Validation"). Print
both comparison tables (baseline + final) and the overall results summary.

### Phase 6: Persist the backlog

Before declaring the run done, update `<skill>/references/improvement-backlog.md`
(create the file if absent). This is non-optional — ceiling findings that exist
only in chat disappear when the session ends.

Write two sections:

1. **Open** — every issue the loop **actually attempted** as a hypothesis and
   could NOT apply in a single iteration (multi-file restructure, author-only
   domain content, flagged-for-review findings from freshen, or rule-ceiling
   discards). For each entry:
   - one-line title
   - dimension number it affects (e.g. "Dim 2" or "Dim 6/8")
   - specific file:line pointer OR the exact file-set that would need to change
   - why skill-improver couldn't apply it in one iteration (e.g. "9-file split",
     "requires author-authored error-handling content", "breaks
     self-consistency without restructure")
   - enough context to act on without re-running the baseline scoring

   **Open is NOT a wishlist.** Hypothetical-future-risk items ("description is
   8 chars from cap, might overflow someday"; "this trigger keyword could
   become ambiguous if X happens") do NOT belong in Open. The bar is: the loop
   proposed this iteration, attempted or planned the mutation, and the
   mutation could not be applied. If you never tried it, leave it out. If
   tomorrow's edits would naturally surface it, leave it out. Open is a
   work-not-done log, not a worry list.

2. **Resolved this pass** — one-line audit of what was fixed. Move items from
   "Open" to "Resolved" if a prior backlog listed them and this run closed them.

   **What "Resolved" means:** the iteration applied a real mutation that the
   metric registered. Creating a placeholder file (e.g., empty `sources.md`
   with no `Last verified:` dates) does NOT resolve a Dim 9 staleness cap —
   the cap stays. Log such cases as Open with action "run freshen mode", not
   Resolved. Hand-waving that "the structure now exists" is theater.

Format: plain markdown, `## Open` and `## Resolved this pass` as top-level
sections. See `references/improvement-backlog.md` patterns from prior runs for
shape — it is intentionally uniform so future skill-improver loops can diff.

If the backlog already exists with items skill-improver chose not to fix this
run, carry them forward into the new "Open" section with a `(carried YYYY-MM-DD)`
marker so staleness is visible.

If the run produced zero ceiling findings (converged cleanly at ≥90/100),
still update the file — strip "Open" to empty and record the final score under
"Resolved this pass" so the file remains a truthful record.

---

## Operating Rules

### Never Stop (Unless Asked)

Run the loop continuously. Do not ask permission between iterations. The user may be away. Print status lines so they can review when they return.

### Git as State Machine

When improving skills in a git-tracked directory:
- Commit each kept improvement individually.
- Use `git diff` to show what changed on discard before reverting.
- The branch tip always represents the best-known version.

### Prioritize Deletion Over Addition

In practice, removing redundant content produces the largest per-iteration score gains. When choosing between an additive improvement (+1 from adding content) and a subtractive one (+1 from deleting content), prefer deletion — it improves simplicity as a side effect.

### One File at a Time

Each iteration targets one file. If the improvement requires touching multiple files (e.g., moving content from SKILL.md to references/), that counts as one atomic change.

**The split test for atomicity.** "Atomic" is not a word — it is a constraint. State the change in 10 words, present-tense, single verb. "Move gotchas section to references/gotchas.md." If the honest sentence needs an "and" — "move content to references/ AND fix second-person AND tighten terminology" — it is three iterations, not one. Pure relocation is allowed; relocation that quietly rewrites prose is not. If during a structural move you find yourself editing a sentence's wording, stop, finish the move with the prose unchanged, score, then propose the prose edit as the next iteration. The reason: bundled iterations attribute the score lift to the wrong cause, which means future loops will pick the wrong category to pivot to.

### Preserve the Author's Intent

The skill reflects the author's domain expertise. Improve structure, clarity, and adherence to best practices. Do NOT rewrite the author's domain knowledge or change what the skill teaches — only how it teaches it.

---

## Blind Validation

Self-evaluation bias is real — the agent that wrote improvements tends to score
them generously. Blind validation uses independent subagents that have never seen
the skill to score it objectively. Run it twice: at baseline and after the loop.

### When to Run

1. **Baseline** — after the self-score in Phase 0 step 4, spawn a blind scoring
   agent in the background. It runs in parallel with the improvement loop.
2. **Final** — after the loop stops, spawn another blind scoring agent on the
   final version.

### Agent Prompt

Spawn a subagent with this task (substitute paths):

```
Score this Claude Code skill for quality. Be honest and critical — most decent
skills score 50-70, 80+ is excellent.

1. Read the rubric: <skill-improver-dir>/references/quality-rubric.md
2. Read the design guide: <skill-improver-dir>/references/anthropic-skill-design.md
3. Read the skill: <target-skill-dir>/SKILL.md
4. Read all files in: <target-skill-dir>/references/
5. Read all scripts in: <target-skill-dir>/scripts/ (if the directory exists)

For Dimension 1: check what falls within the first 1,536 chars of combined
`description` + `when_to_use`, and penalize if key trigger phrases are past the
cutoff. Note whether the skill splits the two fields or stuffs everything into
`description`.
For Dimension 9: check if appropriate frontmatter fields are used.

Score each dimension (0-10) with one-sentence justification. Return the
scoring table, the total, and a "Top 3 issues" list (one line each, with
file:line if applicable).
```

Spawn via whatever subagent mechanism the runtime exposes — in Claude Code,
the `Agent` tool with `subagent_type: general-purpose` and
`run_in_background: true` for the baseline (parallel with the loop), foreground
for the final (comparison table needs the result). If no subagent mechanism is
available, run the same prompt manually in a fresh session and feed back the
result.

**Model selection:** pin the validation subagent to the most capable
model available (Opus 4.6+ as of 2026-05). Boris Cherny's
counterintuitive observation: cheaper-per-token models often use *more*
total tokens on hard tasks because of correction loops, so the
"expensive" model is paradoxically the cheapest path to a reliable
answer. Validation is the loop's hard task — the dim-by-dim
justifications are what make subsequent iterations targetable, and
shallow Sonnet justifications cost more re-runs than they save in
per-token spend. In the `Agent` call: pass `model: "opus"` (or the
current most-capable identifier) explicitly rather than inheriting the
parent's default.

For the baseline agent, copy the original skill to a temp directory first so
the agent scores the unmodified version even if the loop has already started.

### Comparison Table

After each blind agent returns, print a side-by-side comparison:

```
## Bias Check: [baseline|final]

| # | Dimension        | Self | Agent | Gap |
|---|-----------------|------|-------|-----|
| 1 | Trigger Prec.   |  6   |   7   |     |
| 4 | Actionability   |  9   |   7   | +2  |
|   | **Total**       | 81   |  78   |     |

[FLAG] Dimension 4: self-score 2+ higher than blind agent.
Agent says: "Steps 3-4 lack specific commands."
→ Re-evaluate this dimension with the agent's justification in mind.
```

Only flag dimensions where the gap is 2 or more. If no flags, print
"No dimensions with 2+ gap. Scores aligned."

The blind score does not override the self-score. It surfaces potential bias
for the improvement loop to address — a flagged dimension becomes a candidate
for the next iteration.

---

## Batch Mode

To improve multiple skills:

1. Run `scripts/scan-skills.sh` to find all SKILL.md files in scope.
2. Score each skill (baseline only) and print a ranked table.
3. Sort by score ascending (worst first).
4. Run the improvement loop on each, starting from the worst. Cap at 5 iterations per skill in batch mode.
5. Print a final summary table: skill name, baseline score, final score, delta, number of kept changes.

---

## Standalone Evaluation (No Loop)

When the user only wants a quality score without iterating:

1. Read the target skill and `references/quality-rubric.md` from the skill-improver directory.
2. Score all 10 dimensions using the scoring template from the rubric.
3. Print the results table. Highlight the lowest dimension and recommend the single highest-impact improvement.
4. If Dim 9 is capped by sources.md staleness (see rubric §Dim 9), recommend running `freshen <skill>` as the single highest-impact next step.
5. Stop. Do not enter the improvement loop unless asked.

---

## Freshen Mode

Probe a skill's external references for staleness and apply verified updates
in place. Shares the keep/discard loop with the improvement mode but sources
hypotheses from online evidence (release notes, doc commits, deprecation
signals) rather than rubric scores.

### Invocation

- `freshen <skill-path>` — single skill
- `freshen --all` — every skill returned by `scripts/scan-skills.sh`
- `freshen --group <glob>` — subset, e.g., `vllm-*`

Freshen defaults to **apply** — the loop commits verified updates. For a
read-only staleness readout, use Standalone Evaluation (Dim 9 reflects
`references/sources.md` freshness automatically).

### Phase F0: Setup

1. Read the target skill directory (SKILL.md + `references/`).
2. Read `references/freshen-patterns.md` from the skill-improver directory for ref-extraction heuristics and probe templates.
3. Snapshot: `cp -a <skill-dir> /tmp/<skill-name>-freshen-baseline`.
4. Open a findings log: `id | ref | skill-says | current | classification | action`.

### Phase F1: Extract References

Precedence (extractors defined in `freshen-patterns.md` §1):

1. `references/sources.md` rows — authoritative refs with prior `Last verified` / `Pinned` markers.
2. SKILL.md + other reference-file scan — URLs, `owner/repo` patterns, CLI names with versions, semver strings, API paths, dated claims.
3. Deduplicate (normalize URLs, collapse owner/repo variants).

If the target skill has no `sources.md`, create one in Phase F6 from the extracted set so future freshens have a baseline.

Mark rows with `<!-- ignore-freshen -->` to exclude refs the author deliberately keeps as-is (e.g., historical references).

### Phase F2: Probe

For each ref, run the cheapest applicable probe first (templates in `freshen-patterns.md` §2). Stop probing a ref as soon as it produces a finding.

Default probe budget: **20 per skill, 100 per batch run**. On budget exhaustion, stop probing and summarize; flag the skill `partial-freshen` in the log.

### Phase F3: Classify

| Class | Action |
|-------|--------|
| `fresh` | Stamp `Last verified: <today>` on the sources.md row; no content change |
| `version-drift` | Hypothesis: bump pinned version + version-specific guidance |
| `deprecation` | Hypothesis: replace deprecated API / flag with current equivalent |
| `new-feature` | Hypothesis: add a ≤3-line note IFF feature maps to an existing trigger phrase in the skill's `description` / `when_to_use` |
| `broken` | Hypothesis: update or remove the ref |
| `unverifiable` | Leave unchanged; note the ambiguity in the log |

Only drift, deprecation, new-feature, and broken produce mutation hypotheses.

### Phase F4: Mutate (One Finding at a Time)

Same atomicity rule as the improvement loop — one finding per iteration, diff minimal, cause attributable. Always cite the verifying source URL.

### Phase F5: Accept / Revert

Decision rule (different from score-based loop — verification-based):

- **Verified source + ≤ equal complexity** → KEEP. Update sources.md with new `Last verified:` (and `Pinned:` if relevant). Commit per `freshen-patterns.md` §4.
- **Unverified** (single unofficial source, probes ambiguous) → DISCARD. Do not guess.
- **>20 added lines for one finding** → DISCARD and flag for human review in the summary.
- **Breaks self-consistency** (orphans a section, contradicts another part) → REVERT.

### Phase F6: Stamp and Summarize

1. Any ref that probed successfully — fresh or updated — gets `Last verified: <today>` in sources.md.
2. If sources.md was absent at Phase F1, create it now from the successfully-probed refs.
3. Print summary: total findings, kept, discarded, unverifiable, flagged-for-review.
4. Stop. Do not re-probe the same skill in the same session.

### Batch Mode

`freshen --all` iterates skills sequentially:

1. Scan scope via `scripts/scan-skills.sh`.
2. Rank by sources.md staleness (oldest `Last verified:` first; missing dates sort last).
3. Cap findings-per-skill at 5 in batch mode.
4. Share the 100-probe global budget across the batch; stop early on exhaustion.
5. Print ranked summary: skill, findings, kept, new stamp date.

### Anti-Patterns

- Do NOT replace concrete guidance with "see release notes" — extract the specific change.
- Do NOT bump a pinned version without checking the breaking-change section — pins often exist for reasons a diff can't see.
- Do NOT trust a single social-media post — require an authoritative source (official docs, release notes, merged PR, maintainer issue response).
- Do NOT rewrite content unrelated to a finding — each mutation is scoped to its finding.

---

## Trigger Mode

Measure and tune a skill's frontmatter `description` (and `when_to_use`) so it
reliably fires when it should and stays silent when it shouldn't. Same
keep/discard hill-climbing structure as `improve`, but the metric is **trigger
rate against an eval set** — exactly the methodology Anthropic's own
`skill-creator` uses for description optimization (60/40 train/test split,
3 runs/query, blinded test scores, ≤1024-char hard cap).

**Use trigger mode when:** a user reports "the skill didn't fire when I asked
X", "Claude isn't using my skill", or you suspect a description is too vague,
too narrow, too keyword-collision-y, or simply written in the wrong vocabulary
for how users actually phrase requests. Score-mode bumps Dim 1 (Trigger
Precision) on subjective rubric judgment; trigger-mode measures it empirically.

Reference: `references/trigger-patterns.md` for the full pattern catalogue,
eval-set construction rules, decision tree, and worked example.

### Phase T0: Setup

1. Read the target skill (SKILL.md frontmatter, body, references/).
2. Read `<skill>/references/improvement-backlog.md` if present — open
   "trigger" findings carry forward.
3. Read `references/trigger-patterns.md` from the skill-improver directory.
4. Snapshot the skill: `cp -a <skill-dir> /tmp/<skill-name>-trigger-baseline`.
5. Initialize a results log: `iter | train | test | desc-chars | status | change`.

### Phase T1: Build (or load) the eval set

Look for `<skill>/references/trigger-evals.json`. If present, use it as the
starting eval set and append any new user-reported failures from `--missed
"<phrase>"` flags as new should-trigger entries.

If absent, construct a fresh eval set per `references/trigger-patterns.md`
§"Eval-set construction":

- 6–8 should-trigger queries: prioritise user-reported failures verbatim;
  fill the rest with description paraphrases, body-mined examples, and
  everyday user vocabulary.
- 5–7 should-NOT-trigger queries: keyword-collision distractors,
  sibling-skill territory, generic conversation, adjacent-domain decoys.

Save to `<skill>/references/trigger-evals.json`. The file persists so future
trigger-mode runs build on the same eval baseline.

### Phase T2: Probe baseline

Run the probe with a stratified train/test split:

```bash
python3 ${CLAUDE_SKILL_DIR}/scripts/probe-trigger.py \
  --skill-path <skill-dir> \
  --eval-set <skill-dir>/references/trigger-evals.json \
  --holdout 0.4 --runs-per-query 3 --num-workers 6 --verbose
```

The probe writes a synthetic slash-command containing the candidate
description into `.claude/commands/`, runs `claude -p "<query>"` with
`--output-format stream-json --include-partial-messages`, and parses the
stream for a `Skill` or `Read` `tool_use` whose target name matches the
synthetic id. Each query runs N times to measure trigger rate; rate >=
threshold counts as triggered.

Read the JSON output: `train.summary` and `test.summary` carry pass/fail
counts; per-query records carry `trigger_rate` for diagnosing the failure
type.

If the `claude` CLI is missing or unauthenticated, the probe fails fast.
Fall back to manual A/B testing per `trigger-patterns.md` §"Fallback when
`claude -p` is not available" — print the candidate description and the eval
set, ask the user to spot-check from a fresh session. Do NOT use a subagent
to "guess" trigger behavior; the agent will roleplay, not measure.

### Phase T3: Hypothesize

Categorise the train-set failures and pick ONE mutation type per
`references/trigger-patterns.md` §"Mutation patterns by failure type":

| Failure profile | Pattern |
|---|---|
| All failures are should-trigger misses (under-trigger) | T1 — add explicit phrases, be pushier, front-load |
| All failures are should-NOT false-positives (over-trigger) | T2 — add negative boundary, tighten scope |
| Mixed under + over | T3 — fix whichever class has more failures first |
| 1/3 or 2/3 trigger rates dominate | T4 — strengthen redundancy, bump runs-per-query to 5 |
| Cap-bound: description hits 1024 chars | T5 — re-balance into description vs when_to_use |
| Sibling skill steals the trigger | T6 — backlog finding, NOT single-skill mutation |

### Phase T4: Mutate

Apply ONE change to the frontmatter (description and/or when_to_use). Hard
constraints:

- `description` ≤ 1024 chars (Agent Skills spec hard cap; descriptions over
  that are rejected by `skills-ref validate`).
- Combined `description` + `when_to_use` ≤ 1,536 chars (Claude Code listing
  truncation in v2.1.105+; targets older Claude Code use 250).
- Third person, imperative voice ("Use this skill for…", not "You can use…").
- Do NOT touch SKILL.md body — it loads after triggering and cannot influence
  trigger decisions. Trigger mode is frontmatter-only.

### Phase T5: Re-probe and decide

Re-run the probe with the new description (override via
`--description "<text>"` so the file isn't written until accepted).

Decision rule on **train** scores:

- **Train improved by ≥1 query** → KEEP. Write the new frontmatter to
  SKILL.md. New baseline.
- **Train equal but description shorter/simpler** → KEEP (simplification ties
  per the Karpathy rule).
- **Train equal or worse** → DISCARD. Revert the proposal (file unchanged
  since override was used).
- **Train improved AND test got worse by 2+ queries** → DISCARD as overfit.
  The mutation taught Claude the train phrasings without generalising.
- **Train improved BUT description hit the 1024 hard cap** → DISCARD, plan
  T5 next iteration.

### Phase T6: Loop

Up to **5 iterations** (default; trigger probes are 5–10x more expensive
than rubric scoring because each probe shells out to a model). Stop when:

- Train pass-rate ≥ 95% AND test pass-rate ≥ 80% — converged.
- 3 consecutive discards across at least 2 mutation patterns — ceiling
  mapped. Surface what was tried.
- A T6 (cross-skill conflict) finding emerges — single-skill loop can't fix
  it; surface as backlog.
- User interrupts.

### Phase T7: Apply and persist

1. Pick the winner by **TEST** score (NOT train — overfit guard, same as
   Anthropic's loop).
2. Write the winning frontmatter to `<skill>/SKILL.md`. Do NOT edit body.
3. Update `<skill>/references/trigger-evals.json` — append a `last_run`
   metadata block with date, baseline score, final score, iteration count.
4. Update `<skill>/references/improvement-backlog.md`:
   - Move resolved trigger items to "Resolved this pass".
   - Add any T6 cross-skill conflicts as new "Open" items.
5. Print summary table:
   ```
   skill: <name>
   baseline: train X/N, test Y/M
   final:    train X'/N, test Y'/M
   delta:    +A train, +B test
   iterations: I (K kept, D discarded)
   eval set: <skill>/references/trigger-evals.json (saved for next run)
   ```

### Batch Mode

`/skill-improver batch trigger --all` (or `--group <glob>`) iterates skills
sequentially:

1. Scan via `scripts/scan-skills.sh`.
2. Probe baseline on each — rank by `(train_pass_rate * 0.6 + test_pass_rate
   * 0.4)` ascending (worst first).
3. Run trigger loop per skill, capped at 3 iterations in batch mode (probes
   are expensive).
4. Print ranked summary: skill, baseline, final, delta, iterations.

### Anti-Patterns

- Do NOT mutate the SKILL.md body — body cannot influence trigger.
- Do NOT pick the final by train score — always test, to guard overfit.
- Do NOT eval against only passing phrasings — include user-reported
  failures and adversarial negatives.
- Do NOT skip negatives — pure-recall tuning makes the skill grab everything.
- Do NOT run on plugin or managed skills (`~/.claude/plugins/`) — trigger
  mode mutates frontmatter; only personal/project skills are in scope.
- Do NOT run trigger mode in the user's active project — the probe writes
  temp slash-commands to `.claude/commands/`. Use a clean cwd.

---

## Philosophy Mode

Cheap weekly check that runs the three Boris-derived signals as one
pass without spinning up the full 10-dim rubric or the trigger eval set.
Sibling to `freshen` and `trigger`. Sourced from Boris Cherny (creator
of Claude Code, Anthropic; Lenny's podcast 2026). Output is a Boris
score (0-3 anti-patterns flagged) plus the existing dim caps that fire
as a side-effect.

### Invocation

```
/skill-improver philosophy <skill-name>
/skill-improver batch philosophy --all
```

### Phase P0: Setup

1. Resolve the skill (same as Phase 0 of `improve` mode).
2. Read `references/quality-rubric.md` §"Boris Alignment Check",
   `references/freshen-patterns.md` §"4b. Scaffolding Decay Probes",
   and `references/trigger-patterns.md` §"Minimalism test (Boris
   alignment)".

### Phase P1: Run the three checks

| Check | Source | What it flags |
|---|---|---|
| Boris Alignment | quality-rubric §"Boris Alignment Check" | Strict workflow, context dump, model-version compensation |
| Scaffolding Decay | freshen-patterns §4b | Old Claude-version language, prescriptive procedural lists, monolithic context sections |
| Minimalism | trigger-patterns §"Minimalism test" | High-trigger-rate / low-body-content collapse candidates |

Run all three. Each check returns 0 or more findings.

### Phase P2: Score

```
philosophy_score = 3 - count(distinct anti-patterns flagged)
```

Boris score interpretation:

| Score | Meaning |
|---|---|
| 3 | Skill is Boris-aligned. No structural debt. |
| 2 | One anti-pattern. Note in justification, defer if minor. |
| 1 | Two anti-patterns. Flag as ceiling — recommend `improve` mode pass with the flagged dims as targets. |
| 0 | All three anti-patterns. Skill is fighting the model's grain — high probability of decay across the next 1-2 model releases. Recommend a structural rewrite, not iterative improvement. |

### Phase P3: Apply (optional)

Philosophy mode does NOT auto-apply mutations. It surfaces the
findings; the operator decides whether to:

1. Run `improve` mode with the flagged dims as the optimization target,
2. Run `freshen` mode (which now includes scaffolding-decay probes), or
3. Manually rewrite — Boris-score-0 skills typically need that, not
   loop-driven hill climbing.

### Phase P4: Persist

Append a Philosophy entry to `references/improvement-backlog.md` (create
the file if absent) with the score, flagged patterns, and a one-line
recommendation per finding. Same format as the existing backlog
pattern (e.g. `instructions-triage`'s backlog).

### Batch Mode

`/skill-improver batch philosophy --all` runs P1-P4 across every skill
under `~/.claude/skills/`. Output is a leaderboard of Boris scores so
the operator can target the worst offenders first. ~10 seconds per
skill — fast because no probes hit external services and no rubric
re-scoring runs.

### Anti-patterns

- Running `philosophy` mode immediately before a major Claude release
  drop. The bitter-lesson signal is most useful 1-2 weeks AFTER a new
  release lands — that's when scaffolding-decay flags are confirmed
  by current behaviour, not predicted from old behaviour.
- Treating Boris score 3 as a permanent green light. Philosophy is a
  point-in-time check; re-run at least quarterly or after each Claude
  major release.
- Auto-applying philosophy findings. The "right" response to model-
  version compensation is usually deletion, but the deletion needs
  author judgment — the loop should not delete operator-curated rules
  unilaterally.

---

## Additional Resources

### Reference Files

- **`references/quality-rubric.md`** — Full scoring rubric with sub-criteria, examples of each score level, and common failure patterns. Load this before scoring.
- **`references/improvement-patterns.md`** — Catalog of common improvements organized by dimension, with before/after examples.
- **`references/freshen-patterns.md`** — Reference-extraction heuristics, probe templates (gh CLI / WebFetch / WebSearch), and classification rules for Freshen Mode.
- **`references/trigger-patterns.md`** — Eval-set construction, mutation patterns by failure type, decision rules, and worked example for Trigger Mode. Load before running `trigger`.
- **`references/anthropic-skill-design.md`** — Anthropic's skill design practices, complete frontmatter reference, Agent Skills standard, and platform constraints. Consult when scoring Dimensions 1, 2, 8, and 9.
- **`references/sources.md`** — Dated per-URL index of official docs, specs, changelogs, and blog posts. Freshen Mode reads and stamps `Last verified:` / `Pinned:` fields here.
- **`<skill>/references/improvement-backlog.md`** (per-target, not in skill-improver's own dir) — Carries ceiling findings across skill-improver runs. Read in Phase 0 step 3; updated in Phase 6. Each target skill that has ever been through skill-improver should have one.
- **`<skill>/references/trigger-evals.json`** (per-target) — Persistent eval set for Trigger Mode. Built on first `trigger` run; reused and extended on subsequent runs. Schema: `[{"query": str, "should_trigger": bool, "source": str}, ...]`.

### Scripts

- **`scripts/scan-skills.sh`** — Find all SKILL.md files in profile and project scopes. Outputs paths sorted by modification time.
- **`scripts/probe-trigger.py`** — Trigger-mode measurement tool. Adapted from anthropics/skills `skill-creator/scripts/run_eval.py`. Spawns `claude -p` subprocesses against a synthetic slash-command and parses stream-json for `Skill`/`Read` `tool_use` events to compute per-query trigger rate. Supports stratified train/test split, configurable runs-per-query, threshold, and parallelism.