Regression Suite Curator

---
name: regression-suite-curator
description: Curate WHAT belongs in the regression suite and at which tier — promote fixed bugs at or above the chosen severity threshold and decide which tests earn smoke-tier status; document retirement/demotion for dead features, duplicated coverage or negative-value tests; fit smoke/PR/full/nightly tiers to runtime budgets; detect duplicate coverage; enforce quarantine owner, ticket and expiry; assign per-test ownership. Produces evidenced curation decisions; permanent security regression tests retire only through the human-approval path. Use to choose regression/smoke membership, trim a slow suite, formalize bug-fix promotion, clean up quarantine, or meet a runtime budget. Do NOT use to diagnose flakes (flaky-test-detective), design CI/retry machinery (qa-automation-architect), map coverage (test-coverage-mapper), or write tests (engineer skills).
---

# Regression Suite Curator

**Reading key:** PR means pull request; CI means continuous integration; QA
means quality assurance; CODEOWNERS is a repository file that names the owners
of paths.

## Purpose

Keep the regression suite worth its runtime: every test in it earns its
place, every fixed bug that matters is represented, tiers match their
budgets, quarantine is a workflow instead of a graveyard, and nothing
disappears silently. The deliverable is a curation decision list — promote /
retain / demote / retire / quarantine-action per test group, each with
evidence and rationale — plus the standing promotion and retirement rules
the team applies between curation passes.

## Use When

- Use when: asked what belongs in the regression or smoke suite, or which
  tests should gate a release.
- Use when: the suite is slow/bloated and needs trimming against a runtime
  budget without losing protection.
- Use when: formalizing bug-fix→regression-test promotion (the
  regression-first bug-fixing practice, catalog item 190; historical planning
  context, not a current gate) or smoke-tier membership criteria.
- Use when: the quarantine list has become permanent storage — entries
  without owner/ticket/expiry.
- Use when: after an incident, ensuring the incident's regression tests are
  promoted to the right tier.
- Do NOT use when: a test fails intermittently and needs diagnosis —
  `flaky-test-detective` *(manual-only)* (its case reports are curation INPUT).
- Do NOT use when: designing CI mechanics, sharding, retry policy —
  `qa-automation-architect` (it sets tier budgets; this skill fills tiers).
- Do NOT use when: mapping what tests cover requirements —
  `test-coverage-mapper` (its gap list feeds promotion; its theater list
  feeds retirement).
- Do NOT use when: writing the tests being promoted — engineer skills.

## Inputs to Inspect

1. The suite inventory: tests per tier, runtimes, last-failure dates,
   pass-on-retry counts, quarantine tags — from CI history, not memory.
2. Coverage context: `test-coverage-mapper` output (theater tests, gaps),
   duplicate-coverage candidates (many tests through one behavior).
3. Bug/incident history: fixed bugs and whether each has a regression test;
   incident postmortems naming untested paths.
4. Tier budgets and gating rules from the automation blueprint / QA
   strategy.
5. Ownership signals: who maintains what (CODEOWNERS, git blame on test
   dirs).

## Workflow

When runtime budget and named test protection require an owner choice,
define the budget and coverage; explain viable budget, tier, and protection
paths by reason, risk, benefit, and money/setup/upkeep cost or uncertainty.
Recommend a fit from CI evidence before one decision question. Do not treat
that answer as approval to delete tests or weaken a security regression.

1. **Build the evidence table first:** per test/group — tier, runtime,
   failure history (real catches vs flake noise vs never-failed), what
   behavior it guards, duplication candidates. Curation without CI-history
   evidence is taste, not curation.
2. **Apply promotion rules:** every fixed bug of severity ≥ threshold has a
   regression test at the layer that would have caught it (gaps become
   engineer-skill work items); smoke tier = the few tests proving critical
   journeys/invariants, each with a named reason; incident-derived tests
   promoted per postmortem. Rules catalog in
   [references/curation-rules.md](references/curation-rules.md).
3. **Apply retirement/demotion rules with written rationale per test:**
   dead-feature tests (feature removed → test retires WITH the feature's
   removal PR), duplicated coverage (keep the cheapest reliable
   representative, cite the survivor), theater tests (from the mapper —
   retire or repair), never-failed-and-unreachable tests (demote to nightly
   or retire with evidence they can't fail meaningfully). NO silent
   deletion: every retirement names what protection is lost and why that's
   acceptable.
4. **Guard the protected classes:** permanent security regression tests
   (from `multi-tenant-security-tester`, incident fixes) retire ONLY via
   `human-approval-boundary` with written rationale — flake or slowness is
   never sufficient cause on its own.
5. **Enforce the quarantine registry:** every quarantined test has owner +
   ticket + expiry; expired entries get a decision NOW (diagnose via
   `flaky-test-detective` *(manual-only)*, fix, or retire with rationale) — quarantine is a
   queue, not a destination.
6. **Fit tiers to budgets:** with the automation blueprint's budgets,
   assign/demote until tiers fit — by evidence (catch-rate per runtime),
   not alphabetically; record what moved and why.
7. **Publish the decision list + standing rules**, route implementation
   (test writing → engineer skills; deletions → normal review via
   `reviewable-diff-discipline` *(manual-only)*), and set the next curation cadence.

## Output Format

```
CURATION REPORT — <suite/scope> @ <date>
Evidence table: <test/group → tier, runtime, failure history, guards-what,
                duplication candidates>
Decisions:
  PROMOTE  <test/gap> → <tier> — <rule + evidence>
  RETAIN   <test> — <earns place because…>
  DEMOTE   <test> → <tier> — <evidence>
  RETIRE   <test> — <rationale + protection lost + why acceptable>
  QUARANTINE-ACTION <test> — <diagnose|fix|retire + owner/ticket/expiry>
Protected-class check: <security regressions touched? → human-approval path>
Tier fit: <tier → runtime before/after vs budget>
Owner choice (if needed): <explained budget/protection paths, costs/unknowns, pros/cons, recommendation + why>
Standing rules: <promotion + retirement + quarantine rules going forward>
Handoffs: <new tests → engineer skills; flake cases → flaky-test-detective;
          deletions → review via reviewable-diff-discipline>
Next curation: <cadence/trigger>
```

## Validation Checklist

- [ ] A budget-versus-protection choice has a taught comparison and
      recommendation before one question; test deletion keeps its review gate.

- [ ] Every decision cites CI-history evidence, not intuition.
- [ ] Every fixed bug ≥ threshold maps to a regression test or a work item.
- [ ] Smoke tier members each have a named critical-journey reason.
- [ ] Every retirement has written rationale naming the protection lost.
- [ ] Security regression retirements routed through
      `human-approval-boundary` — zero exceptions.
- [ ] Quarantine registry has owner + ticket + expiry on every entry;
      expired entries decided.
- [ ] Tier runtimes fit budgets after decisions.
- [ ] No tests deleted in this pass — decisions route to normal review.

## Gotchas

- "This test never fails, cut it" — never-failing tests include your best
  guards (they prevent the class of change that would fail them); retire on
  unreachability/duplication evidence, not on quietness.
- Trimming by runtime alone deletes slow-but-only protection for hairy
  paths; catch-rate-per-runtime needs the "what does it guard" column.
- Duplicate-looking tests may assert different failure modes of one
  behavior — read assertions (the mapper's rubric) before declaring
  duplication.
- Quarantine-with-expiry silently becomes quarantine-forever the first time
  an expiry passes unenforced — the registry check is the enforcement.
- A curation pass that lands as one giant deletion PR is unreviewable —
  batch retirements small, per `reviewable-diff-discipline` *(manual-only)*.

## Stop Conditions

- No CI history/runtimes are accessible → curation would be taste; get the
  data or reduce scope to the quarantine registry (which needs only its own
  records).
- A retirement candidate is a security regression test or guards a
  compliance obligation → `human-approval-boundary`, always.
- Suite runtime can't fit budgets without losing named protection →
  escalate explained budget/protection paths and a recommendation to the strategy owner
  instead of quietly cutting.
- Asked to also delete the retired tests right now → deletions go through
  normal review as scoped diffs; hand off the decision list.

## Supporting Files

- [references/curation-rules.md](references/curation-rules.md) — promotion
  rule catalog (bug/incident/smoke), retirement evidence standards,
  quarantine registry schema, tier-fit worksheet.
- `evals/evals.json` — trigger + behavior cases.
- `evals/trigger-evals.json` — discrimination within the unit/build/flake/data
  cluster and against `test-coverage-mapper`.