Skip to content
Back to skills

Qa

ASecurity

VERIFY-ONLY QA pipeline for a ticket a dev claims done (the spec is the oracle). Reads the ticket + spec + schema to derive expected behavior → designs 2–5 test cases (exact repro, boundary, whole-screen sanity, read-only DB verify for writes) → self-provisions missing data through the REAL UI flow (write-gated) → runs them HEADED in the browser → collects evidence (named screenshots, annotated images with in-image captions) → cross-checks every ticket claim against an evidence file → machine...

  • 6 stars
  • 0 votes
  • 0 copies
  • 1 view
  • Added September 2, 2026
developmentpythonrustgobashsqlnodetestinggitdatabasesecurity

Works with

  • terminal
  • cli

Security analysis

A100/100

Scanned September 12, 2026

npx -y skills add connorpham/vteam-harness --skill qa --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Qa?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Qa
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/connorpham-vteam-harness-5240d65c/badge)](https://www.skillsdirectory.com/skills/connorpham-vteam-harness-5240d65c)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: qa
description: "VERIFY-ONLY QA pipeline for a ticket a dev claims done (the spec is the oracle). Reads the ticket + spec + schema to derive expected behavior → designs 2–5 test cases (exact repro, boundary, whole-screen sanity, read-only DB verify for writes) → self-provisions missing data through the REAL UI flow (write-gated) → runs them HEADED in the browser → collects evidence (named screenshots, annotated images with in-image captions) → cross-checks every ticket claim against an evidence file → machine gate (evd_check.py) → challenger sign-off → plain-language REPORT.md anyone can read → tracker comment posted once both machine gates are green, ticket transition per the full-auto policy. NO code change, NO fix — verification only."
argument-hint: "<TICKET: VT-nnn | tracker URL> [assignee=<name>] [base=<app URL — overrides config app.url for this run>]"
---

> **Model routing for this tool** (from `model-routing.data.yaml`, snapshot 2026-08-17):
> `frontier` → **fable** · `workhorse` → **opus** · `standard` → **sonnet** · `utility` → **haiku**
> Roles → tiers: ba-challenger: standard · ba-draft: standard · dev-r1: workhorse · dev-r2: standard (high-stakes: workhorse) · dev-r3: workhorse · explore: utility · qa-challenger: standard · sa-background: workhorse · sa-challenger: workhorse · sa-writer: workhorse
> Resolve at runtime: `python3 .vteam/scripts/model_route.py <role> --tool claude-code [--high-stakes]` — high-stakes diffs (review.high_stakes_*) bump dev-r2 to the workhorse tier.
> Spawning a subagent: pass the resolved name as the Agent tool's `model` parameter.

> **Environment:** `app:` is not configured in vteam.config.yaml — env bring-up is
> manual this session, and `app_check.sh` prints `APP: SKIP`. Set `app.start` +
> `app.url` (+ optional `app.health`, `app.open_files`, `app.headed` — schema:
> DESIGN.md §2) to enable the watchable session: headed Chrome via
> `.vteam/scripts/browser.mjs`, health probe via `.vteam/scripts/app_check.sh`,
> editor opening via `.vteam/scripts/open_files.sh`.


# /qa — verify a delivered ticket against the spec (QA lane, no code)

**What this workflow is NOT:** it never edits product code, never fixes anything,
never expands the dev's scope. It answers ONE question with evidence: **does the
app now behave the way the spec says it should, for the requirement this ticket
describes — without breaking the rest of the screen?**

**Immutable principles** (violation = the verification does not count):
1. **The spec is the oracle.** "Done" means the app matches the spec (+ the schema
   for data rules), NOT "matches the ticket prose" and NOT "no error appears".
   Every expected value cites its spec section / schema line. Ticket contradicts
   spec → spec wins; report the contradiction. Spec silent/self-contradictory →
   that TC is BLOCKED and **goes to the BA lane for arbitration** (the BA opens a
   3-condition gap question; per `raci.md` the BA is R for "bug or intended", the
   owner only A when the BA can't answer from the spec either); never invent an
   expected value.
2. **Real test, no shortcuts, no lying.** Every verdict comes from a browser run
   actually performed THIS session against the running app — never from reading
   code, never from an HTTP status, never from a previous run. A step that could
   not run is BLOCKED with a reason + unblock path — never inferred.
3. **Reading code is allowed only to know WHERE to look** (which route, which
   role gate), never to derive the expected value.
4. **Data through the REAL flow only.** Missing data → create it via the app UI
   (login → screen → action). Direct SQL/ORM writes are forbidden as a test path;
   the DB is read-only, used to VERIFY writes/rollbacks. Every write on a shared
   env follows the write gate: **user present in the session → ask before each
   write**; scheduled session with the user absent (at `autonomy.level: full`) →
   proceed without waiting, with the full record: the write block (screen +
   action + values + cleanup path) into the session minutes BEFORE writing, rows
   marked `ZZTEST`, cleaned through the app's reverse flow after V4. No reverse
   flow to clean with → that TC is BLOCKED, nothing is written.
5. **Evidence a stranger understands.** Per TC: `manifest.md` in plain language +
   per-step named screenshots + a box drawn on the exact region with an in-image
   caption (`annotate.py box --label`). `evd_check.py` must be green before
   reporting.
6. **Cross-check before reporting.** Re-read the ticket line by line; map every
   claim → the evidence file that proves it. An unmapped claim = verification
   incomplete.
7. **A fresh challenger signs off.** Before the verdict is final, spawn one fresh
   agent (empty context) briefed with the ticket + verify-sheet + evidence paths,
   whose job is to FALSIFY (wrong account/role? data-shaped diff? boundary
   missed? stale evidence? jargon in REPORT.md?). Record both cards in
   `evd/<TICKET>/debate.md`.
8. **Visible, human-readable process.** One plain-language `▶` line at every
   phase transition. REPORT.md and the final summary pass the bar: **a
   non-programmer reads 2 minutes and understands everything.**

Tempted to infer a verdict instead of running the step? Your excuse is catalogued in `red-flags.md`, next to the gate that catches it.

Tracker recipes: the configured tracker provider. Evidence tooling:
`.vteam/scripts/annotate.py` (box + side-by-side diff) and
`.vteam/scripts/evd_check.py` (machine gate) — use them, don't hand-roll.
External payment/integration environments are SANDBOX — never a real transaction.

---

## V0 — RESOLVE + ANNOUNCE THE TICKET

1. Fetch issue + ALL comments + attachments. Env vars missing → STOP, ask.
2. **Assignee filter.** Default from config (`assignee=` overrides). Different
   assignee → STOP and ask before verifying someone else's ticket.
3. **Status must be verifiable.** In Review / Ready for QA / Done-claimed = the
   target. To Do / In Progress = not delivered yet → STOP: `BLOCKED (not
   delivered)`. Closed → ask whether to re-verify.
4. **ANNOUNCE before any work** (one block); auto-picked ticket → wait for the go:
   ```
   ▶ Verifying <TICKET> — <summary>
     assignee: <name> | status: <status> | screen/feature: <what>
     target: <app URL> (branch: <protected|PR branch — say which>)
   ```
5. **Pin WHAT code is being verified.** Dev's PR merged → verify the protected
   branch. Not merged → check out the PR branch (read-only) and say so. Unclear →
   ask.

## V1 — READ: ticket + spec + schema (understand as QA, not as dev)

**Who is verifying:** read `docs/team/competencies/qa/qa-identity.md` first —
the senior tester this lane runs as (the oracle rule, severity by consequence,
the third outcome) — then `docs/team/roles/qa.md`. Immediately after, read
`docs/team/competencies/qa/qa-requirement-smells.md` and run its three-line
test on every acceptance criterion **before** deriving a single expected value:
a criterion that fails it is `BLOCKED (not testable as written)`, and that is
the verification's first finding, batched to the requirement owner now — not the
developer, and not after a day of testing.

**KB preflight (index-only):** `docs/qa/knowledge-base.md` §0 + INDEX BEFORE
designing anything; open ONLY tag-matching lessons and answer them in the
verify-sheet (cite by TITLE). Scan `docs/qa/known-issues.md` headings — any
divergence found later is deduped against it before being called a bug.

**Read the change ledger BEFORE the shard:** `docs/specs/changes.md` — filter
CH-nn rows touching this ticket's shard/screen. An expected value that ignores a
decided CH is a wrong expected value.

Write `evd/<TICKET>/verifysheet.md` (REPORT.md points to it in the
appendix; it commits with the V7.4 dossier so the link outlives the session):
- Requirement + AC as the ticket states them; the dev's claimed delivery (PR
  link, comment) verbatim.
- **EXPECTED (from the spec, cited)** — quote the governing sections. Data rules
  from the schema (cite model:field).
- **UI EXPECTED (from the design oracle)** — ① design link on the ticket:
  download frames to `evd/<TICKET>/design/`, exact values from node
  data; ② else look the frame up by screen code in the design index. A UI TC
  compares the real screen against it (`annotate.py diff` → `design_vs_app.png`),
  judged at block/color/text level. The dev submitted a committed `fidelity.md`
  → READ it: re-run `node .vteam/profiles/generic/scripts/ui_fidelity.mjs <TICKET>` to confirm the
  numbers still hold on the build being verified, and audit the INTENTIONAL
  deviation lines — does the declared reason stand (real a11y/spec, or an
  excuse)? QA still designs its own boundaries — never take TCs from the dev's
  spec. Design oracle contradicts the spec (verbatim messages/validation) → spec
  wins; record the design deviation.
- **CLAIM (from ticket + dev comment)** — what they say now works.
- Accounts/roles needed per criterion, data needed.
- Ambiguity → ask the user BEFORE spending effort.

## V2 — DESIGN THE VERIFICATION (before touching a browser)

**Load the competencies this verification needs** — from
`docs/team/competencies/qa/INDEX.md` (read the index, not the tree): **every
row whose `Applies` reads `always`, each opened at the step its `Loads at` column
names** — read them off the index, do not work from a list memorised here: this
sentence used to enumerate them and had drifted both ways, naming two rows that are no
longer `always` and missing three that are. `qa-report-writing` loads at V5 for that
reason: it is needed when the report is written, not when the pack is designed. The
token rows load when the ticket matches — `qa-security-probes` on any auth/roles/money/PII/upload
ticket (`label:`/`term:` match). Record the loaded list in the verify-sheet
under **Competencies:**. Their reference tables (full hostile-input, heuristic,
security and checklist catalogues) live under `competencies/qa/reference/` —
open the one row-set the field in front of you needs; never paste a whole table
into the plan. `qa-test-design` governs how the 2–5 TCs below are chosen and
budgeted by consequence:

Plan in the verify-sheet — usually 2–5 TCs. **Every TC carries a `TITLE:` (a
sentence about behaviour — "A quantity of 0 is refused", never "test qty") and a
`KIND:` from the six the gate knows** — `acceptance` · `boundary` ·
`whole-screen` · `write-readback` · `exploratory` · `security`; the pack MUST
hold a `boundary` and a `whole-screen` case, and the folder is named
`TC_<n>_<what_it_proves>` so `ls` reads as a test plan:
- ① **Exact acceptance path** (`KIND: acceptance`) — the criterion as the ticket/spec states it.
- ② **Boundary pair** — the adjacent input that must behave the OTHER way
  (empty/duplicate identifier, insufficient balance, cancelled record…). A
  delivery that overshoots is a bug.
- ③ **Whole-screen sanity** — the rest of the screen/flow still behaves per
  spec, not only the new spot.
- ④ **Write → read-only DB verify** — if the feature writes, verify the row via
  a read-only query AFTER the UI action; and the rollback path if the spec
  defines one.
- **Per TC, the JOURNEY — you are a person using the product, not a script
  hitting routes.** `evd_check` demands these by name, because asking politely
  did not get them written:
  - **AS** — which account and which role is signed in. A verdict with no actor
    is untraceable, and half of all UI bugs are role-shaped.
  - **PRECONDITION** — what must already be true: the record exists, in which
    state, created by whom. Resolve it now (read-only); never trust an id from
    the ticket to still exist.
  - **ENTRY** — where the user *starts* and what they *click* to arrive: sign in
    → which list → which row → which button. **Typing the screen's address
    proves the address, not the product**: if the menu item is missing, the
    permission is wrong, or the row is unreachable, a deep link hides all three.
    Keep the deep link if you like — as a SECOND path, never the only one.
  - **STEPS** — numbered, each one screenshotted, in the order a person does them.
  - **EXPECTED** — spec-cited, and the region to box.
  - **AFTER** — what changed once the action landed: the message, the list row,
    the value **still there after a reload** (a save that dies on refresh is not
    a save), the neighbouring screen that reads the same data.
  - **BACK** — how the user leaves: Back, Cancel, browser back. What state do
    they return to — is the filter preserved, is a half-typed draft lost, does
    Cancel actually cancel? This is where "it works" usually stops working.
- **Data check (read-only):** does the needed data exist? Never trust values in
  the ticket to still exist — resolve a currently-valid key; an empty list from
  stale data = "data moved", not a bug.
- **The boundary value (②) is chosen with `qa-hostile-inputs`** — one value the
  spec refuses + one a real user produces this week; a boundary chosen for
  convenience is a happy-path case with a different number. **Every TC's STEPS
  carries at least one real-user move** (`qa-user-mindset` — Enter not Save,
  double-click, Back after, refresh, paste) and names its `PERSONA:`. When the
  spec is silent, `qa-heuristics` supplies a consistency oracle so the finding
  has a named owner, not a shrug. Write each record to the
  `qa-case-writing` fifteen-second standard — TITLE a behaviour sentence,
  EXPECTED an observable fact with a citation, none of the banned filler words.

## V2b — ENV BRING-UP

- DB up → migrations clean → dev server up: run the configured `app.start` in a
  BACKGROUND terminal (the Environment block above carries the resolved values;
  gates never start servers — they probe).
- **Health proof, machine-quotable:** `bash .vteam/scripts/app_check.sh --wait 60`
  (a `base=` argument on this run → pass it as `--url <base>`). PASTE the
  `APP: UP …` line into the verify-sheet — that line IS the bring-up proof.
  `APP: DOWN` → BLOCKED with the printed unblock path. `APP: SKIP` → the repo
  has no `app:` config: ask the owner for the URL (and propose the `app:`
  section — schema DESIGN.md §2) before any TC runs.
- Resolve test accounts per role (seed data or existing rows, read-only). No
  usable account → V3 (create via the app's real sign-up flow, write-gated).
- Browser: HEADED — a run the user could watch, and the mechanism is named:
  a REAL Chrome window via `.vteam/scripts/browser.mjs`. Preflight it NOW:
  `node .vteam/scripts/browser.mjs check` — missing Playwright is BLOCKED with
  the printed install command, never a silent headless run. (`app.headed:
  never` — unattended shifts — drops only the visibility, never a screenshot.)

## V3 — PROVISION DATA VIA THE REAL UI FLOW (only if V2 found gaps)

- Trace the producing flow (sign-up → login → the screen that creates the
  record). Drive it in the browser like a real user.
- **Write gate (principle 4): user present → ask first, one block per write**
  (screen + action + exact values + cleanup action); **user absent (scheduled
  session)** → write the block into the session minutes BEFORE acting, then
  proceed. Both paths: capture the run under
  `evd/<TICKET>/data_prep/`, mark rows `ZZTEST`, verify by read-only
  SELECT, clean up via the app's reverse action after V4. No reverse flow → that
  TC = BLOCKED, nothing written.
- Producing flow impossible (feature not built, external dependency) → that TC =
  `BLOCKED (missing data)` with what was tried; never fake it.

## V3b — TICKETS WITH NO SCREEN (the non-UI verify branch)

Activates when the ticket **produces no screen**: a migration, a model, a shared
library function, a background process. Without this branch such tickets are
unverifiable by a browser pipeline — and they are the foundation everything else
stands on, where errors are most expensive.

Unchanged principle: **the spec is still the oracle, and QA still RUNS things
itself — never reads the dev's report as proof.** Only the viewport changes: from
screenshots to database state and command output.

Every TC on this branch needs all four, recorded in
`evd/<TICKET>/TC_n/`:

1. **Migrations run on a CLEAN database, not only the dev's.** Create a scratch
   DB → deploy migrations → status clean. A migration that only works on the dev
   machine (old data masking the flaw) is a broken migration.
2. **Spec-mandated data invariants, checked by read-only SELECT.** Cite the exact
   rule id. Record the statements and real results in `db_verify.md`.
3. **The dev's tests must go RED when the behavior is flipped.** Pick one core
   constraint of the ticket, temporarily break it, run the tests, confirm they
   RED, restore, and `git diff` confirms clean. A green test proves nothing; a
   test that reds in the right place does.
4. **A boundary pair QA invents itself, not taken from the dev's tests.** The dev
   tested their understanding; QA probes exactly where that understanding could
   be wrong.

`evd_check.py` accepts a TC without images when its `manifest.md` declares
`TYPE: NON-UI` — but then a verification file is MANDATORY: `db_verify.md`
(read-only data checks) for anything with a database, or `cmd_verify.md`
(real command + real output pairs — test runs, direct invocations) for pure
libraries and CLIs that have no database to check. No images and no
db/cmd verification is not verification. Never stuff command transcripts into
db_verify.md to satisfy the gate — the file names must stay honest.

## V4 — RUN THE VERIFICATION, HEADED (the actual test)

Announce each run in one line (TC / account / what it proves). Walk the journey
you wrote in V2 — from ENTRY, clicking what a user clicks. For each TC:
screenshot every meaningful step as `01_<what>.png`, `02_<what>.png`… — name the
files after what they SHOW (`01_orders_list.png`, not `01.png`), because the
filenames are the first thing the reader sees — then box the region that carries
the verdict. **Required on every executed UI TC, not just failures**: an
unannotated full-page shot makes the reader guess which pixels mattered, and the
caption is what a stranger reads instead of asking you.

**The headed mechanism — one journey script per TC, kept as evidence:** write
`evd/<TICKET>/TC_<n>/journey.mjs` importing `launch`/`shot` from
`.vteam/scripts/browser.mjs`, walking EXACTLY the V2 journey (ENTRY as clicks,
STEPS in order, AFTER including the reload, BACK), calling
`shot(page, dir, n, "<what_it_shows>")` at every meaningful step — the helper
enforces the `NN_<what>.png` naming by construction. `node TC_<n>/journey.mjs`
opens a real Chrome window the owner can watch. The script STAYS in the TC
folder: it is evidence, and the next QA session re-runs the same journey
against a new build instead of re-improvising it.

```bash
python3 .vteam/scripts/annotate.py box \
  --img evd/<TICKET>/TC_<n>/03_result.png --rect X,Y,W,H [--rect ...] \
  --label "TC_<n>: <what this proves / what diverges>" \
  --out evd/<TICKET>/TC_<n>/03_result_boxed.png
```

The box **hugs exactly X,Y,W,H** (a 4-px frame, no padding) so it proves that
region and no other; measure the rect from the screenshot the box is drawn on.
The caption is burned in on a bar **below** the image, word-wrapped in
`project.language` — below, never above, so no pixel coordinate moves. Several
`--rect` flags box several regions under one caption.

Evidence layout (one folder per TC):
```
evd/<TICKET>/
├── manifest.md            # plain language: the requirement + the COVERAGE: block
│                         #   (security: TC_n | n/a — reason · accessibility: …) +
│                         #   the evd_index block (what was tested / which file is
│                         #   from which case) — regenerated, never hand-edited
├── <TICKET>_testcases.xlsx  # V7: the same record as a workbook (generated by
│                         #   xlsx_export.py; ignored by git — regenerate, never edit)
├── debate.md              # V6 cards (verifier + challenger)
├── data_prep/             # V3 runs, if any
└── TC_<n>_<what_it_proves>/     # the folder name IS the first line of the test plan
    ├── journey.mjs        # the headed run itself (browser.mjs launch/shot) —
    │                     #   re-runnable evidence, committed with the dossier
    ├── manifest.md        # the journey, in plain language — gate-checked:
    │                     #   TITLE: Changing a quantity and pressing Save recalculates the total
    │                     #   KIND: acceptance|boundary|whole-screen|write-readback|exploratory|security
    │                     #   RESULT: PASS|FAIL|BLOCKED
    │                     #   AS: staff@demo (role STAFF)
    │                     #   PRECONDITION: order #4102 exists, state PENDING
    │                     #   ENTRY: signed in → Orders list → row #4102 → Edit
    │                     #   STEPS: 1 change qty 2→3 · 2 press Save
    │                     #   EXPECTED (spec §3.2): total recalculates to 450,000
    │                     #   ACTUAL: total shows 450,000, success message shown
    │                     #   AFTER: 'Saved' toast · list shows 3 · survives reload
    │                     #   BACK: Back → Orders list, filter preserved
    ├── 01_*.png …         # per-step shots, named for what they show
    │                     #   (+ *_boxed.png REQUIRED on the verdict step, with
    │                     #    a caption — a red box with no words explains nothing)
    └── db_verify.md       # write TCs only: the read-only SELECT + rows after the action
```

Gate: `python3 .vteam/scripts/evd_check.py --evd evd/<TICKET>
--expect-tcs <N from V2>` green — pass the PLANNED TC count, so "planned 5, ran
1" can go red — before V5.

## V5 — CROSS-CHECK AGAINST THE TICKET

Table in the verify-sheet — one row per claim in the ticket AND the dev's
delivery comment: `claim → TC → evidence file → matches spec?`. Rules:
- A claim with no evidence row = INCOMPLETE — back to V2/V4; do not report.
- Criterion met but a NEW divergence appeared nearby → verdict `NEW-BUG` for that
  finding; describe it with evidence. A finding to report, never to fix here.
- Ticket misdescribed the behavior but the app still diverges from spec →
  verdict on the REAL divergence, note the correction.

## V5b — DRAFT REPORT.md (plain language, so V6 can grade it)

Beside `COMMIT:` and `VERIFIED-AT:`, the report header carries two more anchors
the gate demands: `ENVIRONMENT: <name — url>` (local · dev · stg · prod — a bug
found on staging is not evidence about production) and `ORACLE: <what "correct"
was compared against>` (the spec shard, or `NONE` when nothing written decided).

Write `evd/<TICKET>/REPORT.md` — in `project.language`, black-box
voice, ZERO jargon in the body (no file:line, route names, "oracle" — appendix
only). Bar: **a non-programmer reads 2 minutes and understands everything.**
Fixed template:

```markdown
# Verification report <TICKET> — <verdict, large: PASS / FAIL / PARTIAL / NEW-BUG / BLOCKED / UNCLEAR>
COMMIT: <sha verified — HEAD of the code the TCs ran on; the verdict binds to the code, evd_check requires it>
VERIFIED-AT: <ISO 8601 timestamp of the run — on squash/rebase repos the pinned SHA dies with the branch, and this is the fallback anchor stale_verdict_check compares post-verdict commits against>

## 1. What does this ticket ask for? (told as a user)
On screen <name>, when <the user does what>, the system must <the right outcome —
per the spec>. The dev reports it done (PR <link>).

## 2. How did I check?
| # | What I did (account, screen, action, data) | Expected (per spec) | Actual | Match? |
(plus: where test data came from / how it was UI-created, if applicable)

## 3. Evidence — what to look at, where?
One line each: file path → what that file SHOWS (covering every TC_<n>).

## 4. Conclusion
1–3 sentences: requirement met or not · new findings or none · recommendation
(move to Done / return to dev).
FAIL / NEW-BUG verdicts → one extra line per finding:
**Severity:** Blocker / Critical / Major / Minor (definitions: roles/qa.md) ·
**Origin:** DEV stage (code diverges from AC) or BA/spec stage (AC/spec wrong or
missing — work returns to the BA lane).

## 5. If BLOCKED / UNCLEAR — why, and what's needed?
**Could not verify <what>** because <reason> · **Tried:** <…> · **To unblock:** <who/what>.

## Appendix (for technical readers)
verify-sheet · spec §/schema citations · debate.md · remaining evidence files.
```

## V6 — CHALLENGER SIGN-OFF

1. Write YOUR verdict card into `debate.md` FIRST (verdict + strongest evidence +
   `MY WEAK SPOT:` the one thing most likely to be wrong).
2. Spawn ONE fresh agent (never forked; model `standard` per model-routing) with:
   ticket verbatim + verify-sheet + evidence paths + REPORT.md + **the
   `## Reviewer lens` block of every competency the verify-sheet lists under
   Competencies:** + this instruction: "FALSIFY this verification: wrong
   role/account? data-shaped difference? boundary untested or chosen for
   convenience? a 'unique/normalised' rule never tested across representations?
   an illegal transition tried only through the UI button? evidence stale or not
   showing what the caption says? a FAIL whose severity or origin is wrong? any
   REPORT.md sentence a non-technical reader can't follow? Work the pasted lens
   items one by one. Return your own card." A challenger card that ignores a
   pasted lens item without saying why is returned.
3. Challenger found a hole → run the decisive experiment (a re-run, a SELECT),
   never argue in prose. Append the resolution + `Remaining dissent:` line.
4. Consensus without a run that actually executed = UNCLEAR, never PASS.

**Optional human report (BDD).** Alongside REPORT.md you MAY emit
`evd/<TICKET>/REPORT.bdd.md` — the verdict as Given/When/Then
scenarios a non-programmer scans in a minute (`docs/team/bdd-report.md`;
`bdd_report_check` guards it: every scenario complete, no code-speak, not
rambling). Code details stay in cmd_verify.md, never in a scenario.

## V7 — REPORT + CLOSE THE LOOP

1. Finalize REPORT.md (challenger's readability fixes applied). Re-run
   `evd_check.py` — must be green. Then make the folder introduce itself and
   produce the spreadsheet, because most of the people this verdict is for will
   never open a markdown file:

   ```bash
   python3 .vteam/scripts/evd_index.py  --evd evd/<TICKET>            # index block → manifest.md
   python3 .vteam/scripts/xlsx_export.py --evd evd/<TICKET> --strict  # <TICKET>_testcases.xlsx
   ```

   The workbook is the same record laid out to ISO/IEC/IEEE 29119-3 — a Summary
   that answers *how many defects and how bad is the worst one* on the first
   screen, the cases as a test-case specification, the defects graded by
   severity, a requirement traceability matrix, and an index of every evidence
   file (headers in `project.language`). It **never invents a value**: a field the
   pack lacks prints NOT DECLARED, and `--strict` exits 1 naming each one — fill
   the pack and re-run. **Fix the pack, never the spreadsheet**: it is generated,
   and a hand-edited workbook is a workbook nobody can check against the evidence.
2. **Summary to the user** = REPORT.md content (verdict + V5 table + evidence
   paths + new findings + remaining dissent).
3. **Tracker comment — self-posted (full autonomy):** 3 parts (result + attached
   evidence + notes/recommendation), posted right after both machine gates are
   green (`evd_check.py` + challenger V6). Print the comment verbatim into the
   session minutes.
3a. **Attach QA images to the tracker — mandatory, with read-back** (images live
   outside git, so the tracker is the only place they survive the session):
   `python3 .vteam/scripts/evd_check.py --evd evd/<TICKET> --attach <TICKET>`
   — uploads every TC image (attach the `<TICKET>_testcases.xlsx` alongside —
   it is the artefact a manager will actually open), re-reads the attachment
   list to confirm each filename, and writes a `## TRACKER ATTACHMENTS` section (name · md5 · url)
   into the root manifest. That pointer is what lets a clean checkout reproduce
   the verdict when images are gone from disk.
3b. **Ticket-closing policy (FULL-AUTO at `autonomy.level: full`):** verdict
   **PASS** + PR merged + CI green → transition the ticket to **Done**. Verdict
   FAIL / NEW-BUG → return the ticket to **To Do + `reopen` label** (distinguishes
   returned work from new work — /dev's DoR gate reads this label and points at
   REPORT.md first) with a comment linking REPORT.md + severity.
   **Blocker/Critical** findings → recorded in the minutes as the NEXT dev item,
   before any new ticket (SLA per roles/qa.md). BLOCKED/UNCLEAR → keep In Review +
   record in the minutes. Every closed ticket is listed in the minutes for the
   owner's acceptance review.
4. **Commit the verification dossier (a verdict must be reproducible):**
   REPORT.md, manifest.md (with TRACKER ATTACHMENTS), verifysheet.md, debate.md,
   TC_*/manifest.md, db_verify.md commit via the light path — a docs branch + PR,
   self-merged when CI is green (images stay out of git). QA never touches
   product code, but its own text dossier MUST reach git — a verdict living on
   one machine is a fabricated report.
5. **Cleanup**: reverse V3 `ZZTEST` data via the app; record any residue in the
   manifest.
6. **A finding OUTSIDE the ticket's scope that deserves its own ticket → QA files
   the Bug ticket itself** (full autonomy; the exemption list still waits for the
   owner). Bug template — missing sections make the DoR gate red:
   ```
   [Environment]  <app URL> — branch, test account
   [Steps to reproduce]  1. … 2. … 3.
   [Expected]  per spec §x.y (docs/specs/<feature>.md)
   [Actual]  what happened
   [Severity]  Blocker / Critical / Major / Minor (roles/qa.md)
   [EVD]  evd/<TICKET>/TC_n/… (boxed image, db_verify if a write is involved)
   ```
   Labels: `qa-found` + area; estimate left for BA refinement (noted in the
   ticket). Dedup against known-issues BEFORE filing (principle from V1).
7. **Close the learning loop** — append a lesson (+ INDEX line) to
   `docs/qa/knowledge-base.md`, even for FAIL/BLOCKED runs (or state "no new
   lesson"). Prefer GRADUATION (§0): gate-shaped → `evd_check.py`; cross-screen →
   known-issues; unconditional → this workflow — then delete from KB.

## Definition of Done
- [ ] V0: assignee matches; status verifiable; announced (incl. WHICH code:
      protected branch or PR branch)
- [ ] Expected derived from spec/schema with citations (never from ticket prose or code)
- [ ] Every TC ran HEADED this session against the running app — bring-up proven
      by app_check's `APP: UP` line quoted in the verify-sheet, journeys driven
      through `browser.mjs` journey scripts kept in their TC folders; blocked
      TCs carry reason + unblock path
- [ ] Boundary TC + whole-screen sanity TC ran (not only the happy path)
- [ ] V1/V2: verify-sheet carries **Competencies:** — `qa-identity` +
      `qa-requirement-smells` + every INDEX row matching this ticket; the loaded
      Reviewer lens blocks went into the V6 challenger brief
- [ ] V1: requirement-smells three-line test run on each criterion; untestable
      ones raised as questions to the owner before testing (not after)
- [ ] Writes verified via read-only DB checks; all test data UI-created, per the
      write gate (ask when present / minutes-first when absent), ZZTEST-marked,
      cleaned up
- [ ] V5 table: every ticket/dev claim mapped to an evidence file
- [ ] Every UI TC walked from its ENTRY (a click path, not a typed address); AFTER
      checked incl. survives-a-reload; BACK/Cancel checked
- [ ] `evd_check.py --expect-tcs <N>` green; REPORT.md follows the template
      (with the COMMIT:, VERIFIED-AT:, ENVIRONMENT: and ORACLE: lines), jargon-free body
- [ ] Every TC has TITLE: (a behaviour sentence) + KIND:; the pack holds a
      `boundary` and a `whole-screen` case; folders are `TC_<n>_<what_it_proves>`;
      EXPECTED/ACTUAL are observable values, never a judgement word
- [ ] Root manifest carries the COVERAGE: block (security + accessibility each a
      case or a reasoned waiver) and a CURRENT evd_index block (`evd_index.py --check`)
- [ ] `xlsx_export.py --evd … --strict` green — `<TICKET>_testcases.xlsx` produced
      from the pack, attached to the ticket, never committed or hand-edited
- [ ] Every *_boxed.png hugs the exact region it proves, caption burned in below
- [ ] QA images attached with read-back + TRACKER ATTACHMENTS section in the manifest
- [ ] V1: change ledger filtered for this ticket's shard/screen
- [ ] Failing verdict: every finding carries Severity + Origin (DEV / BA-spec) —
      evd_check blocks otherwise; ticket returned To Do + `reopen`
- [ ] Out-of-scope findings: deduped, Bug ticket filed per template (labels
      `qa-found` + area) or explicitly judged not ticket-worthy
- [ ] The text dossier COMMITTED via the light path — the verdict reproducible from git
- [ ] debate.md has both cards + resolution; the challenger was a real fresh agent
- [ ] NO product code changed; findings reported, not fixed
- [ ] Tracker comment posted after both machine gates; transition per V7.3b
- [ ] KB preflight done at V1; lesson appended at V7
- [ ] Every phase narrated with a ▶ line; all user-facing output passes the
      plain-language bar

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.