The Pensieve

---
name: the-pensieve
description: Instrument the repo for a measurement week after instruction-layer changes — count what actually happens (advisory re-fires, review-loop rounds, subagent bounces, tokens injected per prompt, session outcomes) so the next round of changes is decided by numbers, not vibes. Use after landing /opus-pocus fixes, before/after any Avada Verbosa ablation round, or for "did those CLAUDE.md changes help", "measure Claude's behavior in this repo", "set up instruction telemetry". Produces the counter list, where each number comes from, and the week-end readout format. NOT for application performance metrics — instruction-layer behavior only.
---

# 🔮 The Pensieve

> *"A week of numbers in the basin. Vibes are not telemetry."*

Every other spell changes the instruction layer. This one finds out whether the changes worked.
The alternative — another audit — just re-applies the same judgment that wrote the instructions;
only behavior counts.

## The counters

Instrument what the spells claim to improve. Core set:

| Counter | Needs | What it validates | Source |
|---|---|---|---|
| **Stop/advisory re-fires per session** — same advisory, unchanged state | hooks + transcripts | Finite Recursum's fingerprint fixes | Hook logs, or grep transcripts for the advisory headers |
| **Review-loop rounds per task** — review→fix→re-review depth | transcripts | Loop caps working | Transcripts / task logs |
| **Subagent bounces per branch** — how often incomplete work goes back | agents | Definition-of-done clarity | Dispatch logs, PR/branch history |
| **Tokens injected per prompt by hooks** | hooks | Muffliato Hookus | Instrument emitters to log output bytes; or measure injected blocks in transcripts |
| **Wrong-agent routings** — dispatched then redirected | agents + transcripts | Descriptio Reducio | Transcripts: dispatches followed by a different agent doing the work |
| **Rule citations of stale facts** — model quotes something no longer true | none | Obliviate Fossilium | Review of session outputs; user corrections |
| **User corrections per session** — times the human had to redirect | transcripts | Everything | Transcripts: user messages contradicting the previous assistant turn |
| **Session outcome** — task done / partial / abandoned | none | The bottom line | End-of-session state |

Add repo-specific counters for whatever the audit flagged worst — measure where the disease was.

Read the `Needs` column before promising anything. A repo with no hooks cannot measure advisory
re-fires or injected tokens; a repo with no subagents cannot measure bounces or wrong routings.
Five of these eight have a prerequisite most small repos do not meet.

## Method

1. **Cheap instrumentation only, and confirm the source exists before you promise the counter.**
   A log line in an existing hook, or a grep over transcripts the harness already writes. On
   Claude Code those are JSONL under `~/.claude/projects/<url-encoded-cwd>/*.jsonl` — the path is
   undocumented and version-specific, so **check it** (`ls ~/.claude/projects/*/ | head`) before
   committing to any transcript-backed counter. A counter whose source you never confirmed is
   exactly the unexercised check this book grades elsewhere. If it is not there, drop the counter
   and name which ones you dropped.

2. **Drop every counter whose subject is zero.** A week of **three** real numbers beats a table of
   eight where five sit permanently at 0 and read as improvement. If a counter needs new
   infrastructure, drop it too.
3. **Baseline first when possible.** Counters before the fixes land make the delta real. When
   fixes already landed, the week still gives an absolute floor for the NEXT round.
4. **One week of normal work.** Not a demo session; the numbers must come from real tasks. Note
   confounds (an unusual crunch, a new team member, a model version change mid-week).
5. **No steering to the numbers.** Nobody works differently because the counters run — the
   point is observing the default.

## Bounds

- Scope: instrumentation design and readout. This pass changes no instruction text and proposes no fixes.
- Cap: about 8 counters, and fewer is better. Drop any counter needing new infrastructure, and any whose subject count is zero in this repo — three real numbers beat eight where five are structurally stuck at 0.
- Done when the counter list, its sources, and the readout format are written. The numbers arrive a week later.
- Quote once, at read time; mark anything you cannot re-confirm `UNVERIFIED` rather than re-reading to check yourself.
- See `skills/opus-pocus/references/casting-safety.md` for the full rules.

## The readout

At week's end, one table: counter · baseline (if any) · this week · verdict (improved /
unchanged / worse / can't-tell). Then the decision each number drives:

- Improved → the fix stands; note it as measured in the repo's decision records.
- Unchanged → the instruction text wasn't the bottleneck; candidate for Avada Verbosa.
- Worse → bisect the landed fixes (they're in themed commits — Avada Verbosa's rule).
- Can't-tell → the counter was bad, not the fix; redesign the counter before re-running.

The readout — not another audit — decides whether to run the next ritual, and which spells it
needs. That closes the loop: audit → fix → measure → audit only what measurement indicts.