Consolidation Scan

---
name: consolidation-scan
description: Discover emerging duplicate/similar code patterns via judgment-driven comparison, track them across runs in a ledger, and report only patterns that have stabilized. Report only — no code changes.
---

# Consolidation Scan

Survey the codebase for **emerging, unnamed** duplicate or near-duplicate logic —
the same responsibility implemented independently in two or more places — weight
candidates by how actively their files are being edited, and track them across
multiple weekly runs in a local ledger before proposing anything. This skill makes
**no code changes** — it reads and reports only.

**Scope note — how this differs from `entropy-scan`:** `entropy-scan` is stateless
and deterministic — it checks a fixed set of *named* rules from `references/principles.md`,
each with an explicit `**Detection:**` instruction, and every run stands alone.
`consolidation-scan` is the opposite shape: there is no named rule to check against.
It relies on judgment — an LLM reading and comparing the *purpose and structure* of
code across files and services — to notice duplication that a deterministic checker
can't name in advance. Because a single pass of judgment-driven comparison is noisy
and prone to one-off false positives, this skill never proposes anything from a
single run: it persists what it finds in `state/consolidation-ledger.json` and only
promotes a candidate once it has recurred, with a stable shape, across multiple runs
(Rule of Three — see Step 6-7). Static clone-detection tools (jscpd, SonarQube CPD)
are explicitly not used here — they only catch textual/token clones; this skill's
job is to catch *semantic* duplication that shares no lines at all (e.g. three
independently-written retry-with-backoff implementations).

---

## Setup: Parse Arguments

Before starting, check if any flags were passed:

- `--summary` — print counts to stdout; skip writing `consolidation-report.md`
- `--dry-run` — run the full scan (survey, churn-weighting, fingerprinting, ledger
  diffing) but skip writing `consolidation-report.md` **and** skip updating
  `state/consolidation-ledger.json`. Print everything that would have been written
  to stdout instead. Use this to validate the skill end-to-end without mutating any
  state.

---

## Step 1: Load the Decisions Registry (Suppressions)

See `references/decisions-registry.md` for the shared decisions-registry pattern this
mirrors — the generic entry schema, ownership, and consumption contract that
`.claude/shipwright/consolidation-decisions.md` is one concrete instance of.

1. Check for `.claude/shipwright/consolidation-decisions.md` in the project root.
2. **If it does not exist**, treat this as "no suppressions configured" — the same
   graceful no-op `entropy-scan` uses for a missing `.claude/shipwright/principles.md`
   override. Print: "No consolidation-decisions.md found — no suppressions configured."
   and continue to Step 2 with an empty suppression list. This is expected on every
   run until a sibling task adds that file — it is not an error.
3. **If it exists**, load it and parse its entries. The exact field format is defined
   by whatever process authors that file (a human or a future skill) — read it
   generically: each entry describes (at minimum) a **pattern description** (what
   duplication was accepted as intentional debt) and, optionally, a **revisit
   condition** (a statement of when the acceptance should be reconsidered — e.g. a
   date, an occurrence threshold, or a triggering event). Do not assume a specific
   heading structure beyond "one entry per accepted pattern, with a description and
   an optional revisit condition" — parse defensively and skip entries you can't
   confidently interpret rather than failing the whole load.
4. Build an in-memory suppression list: one entry per parsed accepted-debt pattern,
   holding at least its description and revisit condition (if any).
5. Print the count of loaded suppressions, e.g. "Loaded 3 accepted-debt entries from
   consolidation-decisions.md."

This suppression list is consulted twice later: once loosely during the survey (Step
3, to avoid wasting effort re-describing a pattern already accepted as debt) and once
authoritatively before promotion (Step 7, which is the actual gate).

---

## Step 2: Load the Ledger

1. Look for `state/consolidation-ledger.json` — agent workspace state, git-ignored,
   the same tier as `state/entropy-patrol-last-run.json` and
   `state/error-patrol-ledger.json` (see `error-scan`'s Step 5 for this tier's
   location convention — it sits alongside repo checkouts, not inside the invoking
   repo's working tree).
2. If it does not exist, treat the ledger as empty: `{"lastRun": null, "candidates": {}}`.
   This is a normal first run, not an error.
3. If it exists, read and parse it. Expected shape (full schema:
   `references/ledger-schema.md`):
   ```json
   {
     "lastRun": "<ISO8601 timestamp of previous run>",
     "candidates": {
       "<fingerprint>": {
         "description": "<normalized pattern description, as last recorded>",
         "files": ["<path>", "..."],
         "occurrence_count": <int>,
         "consecutive_stable_runs": <int>,
         "status": "tracking | ready_to_propose",
         "firstSeen": "<ISO8601>",
         "lastSeen": "<ISO8601>",
         "history": [
           { "run": "<ISO8601>", "description": "<description as observed that run>", "notes": "<why-they-look-like-the-same-responsibility note as observed that run>", "files": ["<path>", "..."] }
         ]
       }
     }
   }
   ```
4. Keep the full parsed ledger in memory — every existing entry must be preserved
   and written back in Step 8, whether or not this run's survey touched it.

---

## Step 3: Survey the Codebase (Judgment-Driven, Not a Clone Detector)

This is explicitly **not** an AST/token clone scan. Do not run or emulate jscpd,
SonarQube CPD, or a diff-based similarity tool. Instead:

1. Read broadly across the codebase's source directories (the plugin's own
   `plugins/shipwright/`, and — when scanning a target project rather than this repo
   itself — its own source layout). Favor breadth over exhaustive depth: sample
   representative files across each service/package rather than reading every file
   in full.
2. Group what you read by **purpose**, not by literal text — e.g. "things that retry
   a flaky operation with backoff," "things that validate an env var is set and
   parseable," "things that paginate a REST API via a cursor header." A candidate
   pattern is any responsibility you observe implemented **independently in 2 or
   more places** — independently meaning neither implementation calls or extends the
   other; if both call a shared helper, that's not duplication, it's already
   consolidated.
3. For each candidate, record:
   - A one-to-two sentence **pattern description** (the responsibility being
     duplicated, written generically enough to still apply if a third instance
     appears with slightly different code)
   - The **involved files** (each file/module implementing the responsibility
     independently), with line ranges where practical
   - A short note on **why they look like the same responsibility** (what's
     structurally or behaviorally shared, even if no lines are shared verbatim) —
     retain this note; it is persisted to the ledger's `history` in Step 6 and is
     what future runs compare against to judge shape stability
4. Before finalizing a candidate, loosely check it against the suppression list from
   Step 1 — if a suppression's pattern description clearly matches, you may skip
   spending further effort refining that candidate's description now (it will be
   filtered authoritatively in Step 7 regardless, so this is just an effort-saving
   shortcut, not the enforcement point).
5. **Stay conservative.** This is the highest-uncertainty part of the whole skill —
   prefer fewer, higher-confidence candidates over a firehose of speculative ones.
   A weak or borderline candidate that shouldn't have been flagged pollutes the
   ledger and can falsely accumulate `occurrence_count` over unrelated runs. When in
   doubt, leave it out — a real duplication will keep showing up on its own in
   future runs.

---

## Step 4: Weight Candidates by Churn

For each candidate's involved files, compute a recency/frequency signal from git
history:

```bash
git log -1 --format=%cd --date=relative -- <file>
git log --since="90 days ago" --oneline -- <file> | wc -l
```

- The first command gives how recently the file was last touched (e.g. "3 days ago").
- The second gives how many commits touched it in the last 90 days (a frequency
  signal).
- For a multi-file candidate, take the **most recently touched** file's recency and
  the **sum** of the 90-day commit counts across all involved files as the
  candidate's churn signal.
- If git history is unavailable for a file (e.g. it's untracked or the command
  errors), treat that file's churn as unknown/zero rather than failing the whole
  candidate.

Rank candidates churn-first (most recently/frequently touched first) — this is a
prioritization signal for the report and for triage effort, not a filter. A dormant
candidate is still tracked in the ledger; it just surfaces lower in ordering and is
less urgent, since files nobody is editing don't suffer from the "update the same
logic in two places" pain this skill exists to catch.

---

## Step 5: Compute a Fingerprint and Match Against the Ledger

The fingerprint is the join key between this run's candidates and the ledger's prior
entries. Getting this wrong means the stabilization counter (Step 6) never
accumulates correctly, so keep it conservative and deterministic:

1. **Normalize** the candidate's pattern description: lowercase, collapse
   whitespace, strip trailing punctuation.
2. **Sort** the candidate's involved file paths alphabetically.
3. **Fingerprint** = a stable hash (e.g. SHA-256, truncated to a short hex prefix
   for readability) of the normalized description concatenated with the sorted file
   list, e.g. `sha256(normalized_description + "|" + sorted_files.join(","))`.
4. For each of this run's candidates, look up its fingerprint in the ledger's
   `candidates` map from Step 2.
   - **Match found** → this is a recurrence of a tracked candidate; proceed to
     Step 6 to update its counters.
   - **No match** → this is a brand-new candidate; create a new ledger entry with
     `occurrence_count: 1`, `consecutive_stable_runs: 0`, `status: "tracking"`,
     `firstSeen`/`lastSeen` set to this run's timestamp, and a `history` array
     containing this run's single observation.

**Important limitation to keep in mind:** because the fingerprint includes the
involved file list, a candidate whose set of duplicated files changes materially
between runs (e.g. a third occurrence appears in a new file, or one occurrence gets
renamed/moved) will **not** match the prior fingerprint and will instead register as
a new candidate. This is a deliberate conservative tradeoff for the first version —
it undercounts recurrence rather than risking a false match that corrupts an
unrelated candidate's counters. Note this explicitly when reporting so a human
reviewing the ledger understands why a pattern might appear to "reset." Expect to
revisit this matching strategy after a few real runs.

---

## Step 6: Update Occurrence and Stability Counters

For every fingerprint match found in Step 5:

1. **Always increment `occurrence_count` by 1.** A fingerprint match is by
   definition a recurrence of the same description + file set, so the raw count
   always goes up.
2. **Judge shape stability** before touching `consecutive_stable_runs`. Compare this
   run's candidate description/structure notes (from Step 3) against the ledger
   entry's most recently recorded `history` entry:
   - If the new instance's description and the "why they look like the same
     responsibility" reasoning are **consistent** with the prior recorded shape
     (same responsibility, same general approach — e.g. still "retry a flaky
     network call with backoff," implemented the same way structurally) →
     **increment `consecutive_stable_runs` by 1**.
   - If the new instance's implementation approach has **diverged meaningfully**
     from what was previously recorded (a different algorithm, a materially
     different scope, or the responsibility has forked into something no longer
     well-described by the stored description) → **reset `consecutive_stable_runs`
     to 0**. Note this only fires on an existing fingerprint match — since the
     fingerprint (Step 5) is keyed on the description text, a change large enough
     to alter the *generic* description itself (e.g. "retry with exponential
     backoff" becoming "retry with a fixed delay and no backoff") would already
     produce a different fingerprint and register as a new candidate in Step 5,
     never reaching this branch. What lands here instead is a matched fingerprint
     (same generic description, same files) whose recorded "why they look like the
     same responsibility" reasoning has drifted — e.g. the description still reads
     "retry a flaky network call with backoff" but this run's notes show the
     backoff calculation changed from exponential-with-jitter to a hardcoded fixed
     delay, a structural change not reflected in the (still-accurate-enough)
     generic description. Rule of Three applies to the *shape* stabilizing, not
     just the raw count — a candidate that keeps reappearing but keeps changing
     shape is not ready to propose a single canonical replacement for, because
     there's no settled target shape yet to propose.
   - This is a judgment call, not a mechanical diff — use the same reading
     comparison approach as Step 3. When genuinely unsure whether a change counts
     as divergence, treat it as divergence (reset) — the cost of under-promoting is
     low (it just tracks one more run), while the cost of over-promoting a still-
     shifting pattern is a report entry proposing a canonical shape that's already
     stale.
3. Append this run's observation (description, files, notes, timestamp) to the
   entry's `history` array — never truncate or overwrite prior history entries.
4. Update `lastSeen` to this run's timestamp and `files` to this run's observed file
   list (the most recent observation, for report display — `history` retains the
   full trail).

---

## Step 7: Promote to `ready_to_propose`

For every candidate (new or matched) after Step 6's updates, evaluate promotion:

1. A candidate is promoted to `status: "ready_to_propose"` only if **all** of the
   following hold:
   - `occurrence_count >= 3`
   - `consecutive_stable_runs >= 2`
   - The candidate is **not suppressed** by the decisions registry loaded in Step 1
     — i.e. no accepted-debt entry's pattern description matches this candidate's
     description, *or* it matches but the suppression's revisit condition has been
     met (e.g. a stated revisit date has passed, or a stated occurrence threshold
     has now been exceeded). If a suppression matches and its revisit condition has
     **not** been met, the candidate stays suppressed regardless of how high its
     counters are.
2. If all three hold, set `status: "ready_to_propose"` on the ledger entry.
3. Otherwise, the entry's `status` stays `"tracking"` — it is **silently tracked**,
   not reported. This is the core "watch, don't act immediately" behavior this
   system exists to implement: a candidate can sit at `occurrence_count: 5` with
   `consecutive_stable_runs: 0` indefinitely (kept reappearing, but never in a
   settled shape) and it will never be promoted until its shape stabilizes for two
   consecutive runs.
4. A previously-`ready_to_propose` entry that becomes newly suppressed (a
   suppression was added since it was promoted) should be demoted back to
   `"tracking"` — the decisions registry is authoritative and always re-checked
   fresh each run, never cached from a prior promotion decision.

---

## Step 8: Write the Ledger

Skip this step entirely if `--dry-run` was passed.

1. Build the full ledger content — every entry from Step 2's loaded ledger, with
   this run's new/matched entries merged in (updated counters, updated `status`,
   appended `history`). Entries that existed in the prior ledger but had no
   corresponding candidate this run are preserved unchanged (a candidate not
   observed this run is not evidence it's gone — a survey is a sample, not
   exhaustive; do not delete or decay entries for a miss in a single run).
2. Set `lastRun` to this run's current UTC ISO-8601 timestamp.
3. Overwrite `state/consolidation-ledger.json` with this content — a full replace,
   the same "current-state snapshot" convention `error-scan` uses for
   `state/error-patrol-ledger.json` (not an append-only log like `entropy-scan`'s
   quality log).
4. Print: `Ledger updated: state/consolidation-ledger.json`

---

## Step 9: Write the Report

Skip this step if `--summary` or `--dry-run` was passed.

Write `consolidation-report.md` to the project root (overwrite if it exists). It
lists **only** entries with `status: "ready_to_propose"` after Step 7 — entries
still `tracking` never appear here, no matter how high their `occurrence_count` is.
Format:

```markdown
# Consolidation Report

**Generated:** {YYYY-MM-DD HH:MM} {timezone}
**Decisions registry:** {".claude/shipwright/consolidation-decisions.md" | "none found — no suppressions configured"}
**Candidates ready to propose:** {count}
**Candidates still tracking (not shown below):** {count}

---

## Ready to Propose

{If none: "No candidates have stabilized enough to propose yet. Run /consolidation-scan again in future weeks to keep building history."}

{For each ready_to_propose entry:}
### {short pattern description}

**Fingerprint:** `{fingerprint}`
**Occurrences:** {occurrence_count} · **Consecutive stable runs:** {consecutive_stable_runs}
**First seen:** {firstSeen} · **Last seen:** {lastSeen}

**Occurrences (most recent observation):**
- `{file_path}:{line_range}` — {brief note on this instance}
- `{file_path}:{line_range}` — {brief note on this instance}

**Proposed canonical shape:** {one paragraph describing the shared responsibility and a plausible single implementation/interface that would replace all occurrences — a starting point for a human or a follow-up task, not a mandate}

**Stability history:**
{One line per history entry: run timestamp → description observed that run}

---

{Repeat for each ready_to_propose entry, churn-weighted order (most recently/frequently touched files first, per Step 4)}

---
_Tracked-but-not-yet-stable candidates are not listed here — see `state/consolidation-ledger.json` for the full ledger. This report only reflects candidates that met occurrence_count >= 3 and consecutive_stable_runs >= 2._
```

Rules:
- Only `ready_to_propose` entries are ever written to this file — this is the
  primary behavioral contract of the skill.
- Each run fully overwrites `consolidation-report.md` — the ledger, not old
  reports, is the historical record.

---

## Step 10: Print Summary

Whether or not `--summary` was passed, always print a summary to stdout after the
scan:

```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CONSOLIDATION SCAN COMPLETE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  New candidates tracked this run:   {N}
  Candidates promoted this run:      {N}   (ready_to_propose)
  Total tracked, not yet ready:      {N}   (status: tracking, in ledger)
  Suppressed by decisions registry:  {N}

{If any candidates were promoted this run:}
  Report written to: consolidation-report.md

{If zero candidates were promoted this run:}
  ✓ No candidates ready to propose yet — {N} still accumulating stability.

{If --dry-run: "Dry run — no files written."}
{Else if --summary: "Summary only — consolidation-report.md not written; ledger still updated."}

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```

---

## Constraints (Do Not Violate)

- **No code changes.** This skill reads and reports only. The only files it writes
  are `consolidation-report.md` (project root) and `state/consolidation-ledger.json`
  (agent workspace state) — and neither is written in `--dry-run` mode.
- **No git operations.** Do not commit, branch, or stage anything. `git log` calls
  are read-only history inspection for churn-weighting.
- **No PR creation and no task-store writes.** This skill never queues tasks —
  that is the `consolidation-fix` skill's job (the `entropy-fix` counterpart for
  this scan). `consolidation-scan` only writes `consolidation-report.md`; queueing
  `ready_to_propose` entries as task-store tasks happens in a separate, later run
  of `consolidation-fix`, never inline here.
- **Not a clone detector.** Never substitute a token/AST similarity tool or a diff-
  based heuristic for the judgment-driven comparison in Step 3 — that changes what
  this skill catches and defeats its purpose.
- **Multiple runs before proposing, always.** Never write an entry to
  `consolidation-report.md` on its first observed occurrence — the
  `occurrence_count >= 3 AND consecutive_stable_runs >= 2` gate in Step 7 is not
  optional and does not have a "confidence override."
- **The decisions registry is authoritative and always re-checked live.** Never
  cache a past suppression decision — an entry's suppressed/promoted state is
  re-evaluated fresh every run against the current
  `.claude/shipwright/consolidation-decisions.md` contents.
- **Missing decisions file is not an error.** Same graceful-degradation contract as
  `entropy-scan`'s missing `principles.md` override — scan and report normally with
  an empty suppression list.
- **Ledger is a snapshot, not a log.** `state/consolidation-ledger.json` is fully
  overwritten each run with current state (all entries, not just this run's), same
  convention as `error-scan`'s ledger — except `history` arrays within each entry
  are themselves append-only and must never be truncated.
- **Never silently drop a prior ledger entry.** A candidate not observed in this
  run's survey stays in the ledger unchanged — a single miss is not evidence of
  resolution.
- **`--dry-run` mutates nothing.** No report write, no ledger write — everything
  that would be written is printed to stdout instead.
- **`--summary` skips only the report file**, not the ledger update — the ledger
  must stay current so the next run's occurrence/stability counters are correct
  regardless of which flag was used.
- **No new coupling.** Everything this skill reads or writes stays inside
  `plugins/shipwright/` (this skill file and its references), `.claude/shipwright/`
  (project-level config, read-only), and `state/` (agent workspace state) — no
  external service dependency, no hardcoded repo name.