Core Skill Eval

---
name: core-skill-eval
description: Use only when the user explicitly invokes this skill to evaluate a skill with representative cases, isolated runs, independent judging, and evidence-driven edits.
metadata:
  version: "1.1"
  updated: "2026-07-26"
---

# Skill Eval

Use this only when the user explicitly invokes this skill. Do not run it for
ordinary tasks or start it on your own after a skill change: it needs a target
skill, representative cases, and a clear standard for judging each case.

## Required Inputs

Do not start until these inputs exist:

- **Target skill**: a real `SKILL.md` that can be read.
- **Representative cases**: at least one realistic prompt, screenshot, file, diff, log,
  or other input artifact. Do not invent cases or guess intent.
- **Case bar**: what good looks like and which smells would fail it. For
  judgment-heavy skills, keep the bar at outcome level rather than listing the
  exact answer. For conformance skills, exact criteria are appropriate. Ask when
  the distinction is unclear.

If the user only says "this skill often misses X", first help them turn that
complaint into concrete cases and bars before assigning runners.

## Workflow

1. **Validate inputs and state the governing standard.** Read the target skill and
   identify its purpose, trigger, done condition, and expected failure modes.
   Decide whether the eval is judgment-based or conformance-based. Continue only
   when each case has concrete input and a judgeable bar.
2. **Run isolated cases.** For each case, start a clean runner with only the case
   material and the target-skill invocation. Keep the bar, smells, other cases,
   and evaluation intent out of that run. If files must be written, restrict the
   runner to a throwaway directory. After each run, inspect the live checkout and
   clean accidental leaks.
3. **Judge independently.** Give a separate judge the artifact, the bar, and the
   target skill's purpose. Do not ask the judge to "make it pass." Require
   evidence for every verdict, such as a quote or artifact pointer.
4. **Handle variance.** Re-run important or borderline cases enough times to
   report a pass rate instead of a single anecdote.
5. **Classify misses.** A miss can be a skill defect or a bad case/bar. Name the
   defect before editing: premature completion, vague done condition, missing
   rule, weak trigger, no-op instruction, duplication, sediment, sprawl, or
   missing reference split. Do not encode a bad bar into the skill.
6. **Edit only for the observed defect.** Tighten the done condition, trigger,
   rule, behavior anchor, or reference split that caused the miss.
7. **Re-run the suite.** A fix for one case must not regress another. Stop only
   when the cases clear their bars or when you can explain why the skill cannot
   support a case.

## Output

```text
Target skill:
Cases:
Per-case pass rate or verdict:
Failures:
Defect mapping:
Edits made / proposed:
Re-eval result:
Residual risk:
```

Every verdict needs traceable artifact evidence. A single numeric score is not
enough.

## Do not

- Do not revise a skill to satisfy a case whose bar is wrong.
- Do not compare artifacts against each other when the bar is the standard.