Refusal Census

---
name: refusal-census
description: Attributes each abstention or failure in a run to the internal code site that refused it, from per-input refusal logs, and ranks those sites by sole-blocker count (the inputs that only one site blocks) to set the build order. Use when the question is "why did we abstain/fail on these", "which site is blocking the most inputs", or "what should we build to raise coverage" — and before scoping any coverage work. Triggers on a run whose failures all carry one undifferentiated reason code, and on a build order ranked by touched-count instead of measured sole-blocker count. Not for grouping failures by a structural feature of the input (use cluster-failures, which this cross-tabs against).
---

# Census the refusals

**A run record alone cannot tell you why your pipeline abstained or failed.** A common
failure mode: every abstaining row carries the identical triple (e.g. `tier=T5,
source=abstain, reason=UNNAMEABLE`) — one mechanism label, 100% of rows, zero resolution.
You cannot rank build order from that alone, and prioritisation stalls until you can.

The information usually exists — it is just not in the run record. Most rule- or
pipeline-based chemistry tools log a refusal/reject event per internal decision point (a
template that didn't match, a candidate that failed a constraint, a ring system with no
handler, a functional group with no rule, …) even when the summary metric collapses them
all into one bucket. If your project has such logs and a parser for the codes in them,
this skill connects the two. If it doesn't yet, build the parser first — usually a small
regex/groupby over structured log lines, not new instrumentation.

## The procedure

Copy this checklist and track your progress:
- [ ] 1. Per-row log captured on the corpus, split and tier the goal is defined over
- [ ] 2. Parsed to codes (reason optional), mapped to sites, deduplicated per input; ≥90% of
  failing rows carry a code
- [ ] 3. Ranked by sole_blocker, with first_refusal_site and touched_sites beside it
- [ ] 4. Cross-tabbed against the `cluster-failures` clusters
- [ ] 5. Mean sites per failing input stated; corpus, tier, commit and row count cited

If Step 1 or 2 fails, report UNUSABLE (see Refusal conditions); once the capture is fixed,
return to Step 1.

**1. Capture the log, per row.** Run your pipeline with its logger captured at
WARNING-and-above (or whatever level carries refusal events), one record per input. If a
hung input can hang the whole batch, run in short-lived worker subprocesses with a
per-item timeout so one bad input costs one slot, not the whole run.

**2. Parse to codes** with your refusal-code parser. Three behaviours worth building in
deliberately, seen to matter in practice:
- treat any structured "reason" field as **optional** — real sites often log a bare code
  with no reason attached, or one with a non-numeric suffix. Requiring a well-formed
  reason string silently erases those sites from the census.
- **deduplicate per input.** A code firing five times while the pipeline probes five
  substructures/candidates for one input is still *one* blocker for that input. The
  question is "how many inputs does this site block", not "how often does it fire" —
  un-deduplicated counts mis-rank the build order.
- **map codes to sites.** A site that logs several codes counts once per input.

**3. Rank by sole blocker; report the other two rankings beside it.**

| ranking | meaning |
|---|---|
| **sole_blocker** | inputs where this was the *only* site that fired (after deduplication) — what fixing this site alone flips. **This ranking sets the build order.** |
| **first_refusal_site** | the site that fired *first* for each input — the proximate blocker |
| **touched_sites** | every site that fired for the input — total exposure |

These differ substantially and answer different questions. (Illustrative shape from a
real census: one site was the first-fired blocker on roughly 55% of failing inputs and
touched about 68% of them; a second site touched more inputs overall — about 64% — but
led on far fewer.) Build in sole-blocker order, because only those inputs flip when that
one site is fixed. Keep first_refusal_site and touched_sites as extra information: the
first-refusal count names the proximate blocker, and the touched count tells you what else
will still block those same inputs afterwards, so both set expectations, not build order.

**4. Cross-tab against the structural feature clusters** from `cluster-failures` —
scaffold class, charge state, stereocenter count, size band, functional-group class. A
site that blocks one cluster is a different piece of work from a site that blocks all of
them. Also split each site's count into candidate-availability (nothing valid was
generated) and candidate-selection (a valid candidate lost); they need different fixes.

**5. Report the mean sites per failing input.** If it's, say, 3–4 (median 4), most
failures are blocked at several sites simultaneously, so fixing the leader alone will not
flip most of them to success (its sole-blocker count is how many it does flip). Ranking
without this number over-promises the win — say it up front, before anyone commits to a
build order.

## Refusal conditions

Report **UNUSABLE** and stop if:

- fewer than ~90% of failing rows carry ≥1 code — the capture is broken, not the
  pipeline.
- the census was run on a **different corpus, split, or tier** than the one the goal is
  defined over. This is a real trap: a census computed on one dataset/commit/tier does
  not transfer to another — re-measure on the target split before sizing any build
  against it.

## Related

- `check-target` — validate the class before scoping it.
- `spy-site` — prove the site is actually on the execution path before editing it.
- `cluster-failures` — the structural feature clusters to cross-tab against.
- `reuse-before-rerun` — look for an existing census of the same corpus, tier and commit
  before capturing a new one.
- Your project's own fix-methodology / engineering-log doc — verify the capture/harness
  actually ran before trusting any percentage (a silently-broken capture reports a fake
  100% or a fake 0% just as convincingly as a real one).