[QA Method] Generate agent-native test cases in enriched CSV format from JIRA tickets, features, checklists, or existing suites. Uses business logic invariants and edge case library.
Scanned 9/20/2026
Install to Claude Code
npx -y skills add VirtoCommerce/vc-mcp-testing-module --skill qa-test-cases-generator --agent claude-codeInstalls into .claude/skills of the current project.
Are you the author of Qa Test Cases Generator?
Add the live security badge to your README — it updates automatically with every re-scan.
[](https://www.skillsdirectory.com/skills/virtocommerce-qa-test-cases-generator)More formats (shields.io, HTML) on the badges page.
---
name: qa-test-cases-generator
description: "[QA Method] Generate agent-native test cases in enriched CSV format from JIRA tickets, features, checklists, or existing suites. Uses business logic invariants and edge case library."
argument-hint: "VCST-XXXX | domain | suite ID | migrate <suite> | from-checklist <domain>"
---
# /qa-test-cases-generator — Agent-Native Test Case Generation
Generate structured test cases in the enriched CSV format defined by `test-case-template.md`. Produces test cases that AI agents can execute directly via MCP browser tools — with typed steps, explicit assertions, cross-layer checks, and failure signals.
## Usage
```
/qa-test-cases-generator VCST-4565 # Generate from JIRA ticket (auto-detects layers)
/qa-test-cases-generator cart # Generate for a domain (all layers)
/qa-test-cases-generator suite 06 # Extend existing suite with new cases
/qa-test-cases-generator migrate 04 # Migrate legacy suite to enriched format
/qa-test-cases-generator from-checklist payment # Generate from domain checklist items
/qa-test-cases-generator from-bdd "Given user has items in cart, When they apply coupon..."
# Layer-specific generation
/qa-test-cases-generator VCST-4565 --layer api # REST API tests only
/qa-test-cases-generator VCST-4565 --layer graphql # GraphQL xAPI tests only
/qa-test-cases-generator VCST-4565 --layer admin # Admin UI tests only
/qa-test-cases-generator VCST-4565 --layer e2e # E2E cross-layer flows only
/qa-test-cases-generator VCST-4565 --layer all # All layers (default for tickets)
/qa-test-cases-generator coupons --layer api,graphql # Multiple specific layers
```
## Supporting Files
- **test-case-template.md** — Enriched CSV column spec with step type tags, assertion predicates, cross-layer checks, failure signals. **Read this first — it is the format contract.**
- **test-case-examples.md** — Concrete examples per layer (REST API, GraphQL, Admin UI, E2E). Read when you need a reference output for a specific layer.
## Cross-Agent Knowledge (`knowledge/`)
Load these references during generation:
- **business-logic.md** — `BL-*` invariant IDs to populate `Business_Rule` column
- **e-commerce-edge-cases-library.md** — `ECL-*` IDs to populate `Edge_Case_Refs` column
- **catalog.md**, **store-settings.md** — Product types, store config for realistic test data
- **platform-patterns.md** — Common platform behaviors to inform assertions
- **For any Storefront/UI case — these were previously invisible to this skill and are what make a UI assertion strong:**
- **business-logic.md Domain 15 (`BL-UI-*`)** — the measurable UI invariants (CLS, spacing grid, state-induced shift, content boundary, alignment, touch target, keyboard operability), each with its own `Verify` recipe. Assert them with the measurable tags, never in prose.
- **oracles/critical-ui-scope.md** — 36 components × applicable `BL-UI-*`, each with a per-component audit protocol and **real selectors**. Use it as the scope list: which components this change touches, and which invariants apply to each.
- **skills/qa-design/SKILL.md §State-Stress Pass** — the seven states every UI surface must survive (loading · empty · error/disabled · overflow · validation-error · anonymous · dark theme). This is a ready-made case matrix; sweep it rather than inventing states.
- **automation/storefront-selectors.md** + `scripts/lib/storefront-selectors.generated.ts` — real `data-test-id` values. Prefer a test id over a label (a label is an i18n key, so a label-based locator is locale-dependent — and the language selector is itself under test).
- **skills/qa-sbtm/modern-web-attack-surface.md** — the `UIP-*` probe catalog (§Unexpected-action sweep below).
## Execution
### Step 0: Load the Template
Read `test-case-template.md` from this skill folder. This defines all 15 CSV columns, type tags, assertion tags, and writing guidelines. Every generated test case MUST conform to this format.
### Step 1: Determine Input Source
| Argument | Source | Action |
|----------|--------|--------|
| `VCST-XXXX` | JIRA ticket | Fetch via Atlassian MCP → extract AC, scope, affected modules |
| `domain` | Domain name | Match to domain checklist (`/qa-checklist`) → derive cases from items |
| `suite NNN` | Existing suite | Read `regression/suites/{Frontend\|Backend}/<module>/NNN-*.csv` (module-subdir layout) → identify gaps → generate new cases |
| `migrate NN` | Legacy suite | Read legacy CSV → transform each row to enriched format |
| `from-checklist domain` | Checklist | Read domain checklist → generate 1 case per item (add a second only if the item has a distinct boundary or negative dimension with a concrete bug hypothesis) |
| `from-bdd "Given..."` | BDD scenario | Parse Given/When/Then → map to Steps/Assertions/Preconditions |
### Step 1.5: Determine Target Layers
If `--layer` is specified, use it. Otherwise, auto-detect from the feature scope:
| Feature Signal | Layers to Generate |
|---------------|-------------------|
| New REST endpoint or module API | `api` + `admin` (if has UI) + `e2e` |
| New GraphQL query/mutation | `graphql` + `e2e` |
| Admin UI feature (CRUD, blade, grid) | `admin` + `api` (for data verification) |
| Storefront feature (cart, checkout, catalog) | storefront (base format) + `graphql` + `e2e` |
| Cross-cutting feature (coupons, pricing, orders) | **all layers** |
| Bug fix | layer where the bug was found + `e2e` regression |
**Layer resolution rules:**
1. Read JIRA ticket labels, component field, and affected modules
2. Check if the feature has REST endpoints → include `api` layer
3. Check if the feature has GraphQL operations → include `graphql` layer
4. Check if the feature has Admin UI → include `admin` layer
5. If the feature spans storefront + backend → include `e2e` layer
6. **Default for `--layer all`:** generate for every applicable layer
Each layer produces its own test case block with layer-appropriate tags from `test-case-template.md` (see "Layer-Specific Formats" section).
### Step 2: Gather Context
1. **Identify affected domain(s)** — map input to one or more of the 63 domains in `/qa-checklist`
2. **Load business rules** — read `business-logic.md`, find all `BL-*` invariants relevant to the domain
3. **Load edge cases** — read `e-commerce-edge-cases-library.md`, find all `ECL-*` patterns for the domain
4. **Check existing coverage** — read the target suite CSV (if it exists) to avoid duplicating existing test cases
5. **Get UI context** — read `knowledge/domain/sitemap.md` for page URLs, product types, navigation paths
### Step 2.5: GraphQL Schema Validation (Required for `--layer graphql` or GraphQL-related features)
**MANDATORY** when generating GraphQL test cases. Skipping this step produces invalid queries/mutations.
> **Authoring contract:** new GraphQL test cases MUST follow the runner-native format consumed by `scripts/graphql/graphql-runner.ts`. Read **`knowledge/api/graphql-test-cases-runner.md`** before writing any GraphQL row — it defines the canonical `Steps`/`Assertions`/`Cleanup` grammar (`[AUTH]/[GQL-OP]/[GQL-VARS]/[GQL-EXEC]/[GQL-CAPTURE]/[REST-OP]/[REST-EXEC]/[REST-CAPTURE]/[REST]` + `[ERRORS]/[DATA]/[NULL]/[COUNT]/[VAR]`), `getByPath` filter syntax, `@td()` resolver, capture chaining, common failure modes, and an authoring checklist. Gold-standard examples: `regression/suites/Backend/graphql/050i-graphql-configurations.csv` (CFG-GQL-001…032).
1. **Read schema reference** — read `knowledge/api/graphql-schema.md` (introspected from live endpoint)
2. **Check schema freshness** — if the feature involves new/changed GraphQL operations, run `npm run schema:refresh` first to update the reference from live introspection
3. **Validate every query/mutation** in the test case against the schema:
- **Query/mutation name exists** in the schema (e.g., there is NO `createCart` mutation)
- **Argument names and types match** (e.g., `products` uses `query:`, not `keyword:`)
- **All mutations use `command` wrapper**: `mutation { name(command: { ...fields }) { ...return } }`
- **Input type fields match** — check `InputAddItemType`, `InputCreateOrganizationType`, etc. for valid field names (e.g., `InputCreateOrganizationType` has NO `storeId`)
- **Response field names match return type** (e.g., CartType has flat `subTotal`, not `totals { subTotal }`)
- **MoneyType uses `{ amount currency { code } }`**, not `{ amount currencyCode }`
- **Facets use `term_facets { terms { ... } }`**, not `facets { values { ... } }`
- **Required args (`!`) are always provided** (e.g., `pages` requires `keyword!`)
4. **If the schema reference is missing or stale**, introspect directly:
```bash
curl -sk "{{BACK_URL}}/graphql" -H "Content-Type: application/json" \
-d '{"query":"{ __type(name: \"TypeName\") { fields { name } inputFields { name } } }"}'
```
### Step 3: Derive Test Cases (Minimum Effective Set)
**Guiding principle — quality over quantity.** Every generated test case must have a clear **bug hypothesis**: a specific failure mode it is designed to catch. If you cannot answer "what real bug would this catch and why would it occur?", do not generate the case. Coverage numbers are vanity metrics — a suite of 10 targeted cases that each have a distinct failure hypothesis is more valuable than 50 shallow cases that repeat the same happy path with minor variations.
For each requirement/checklist item/BDD scenario:
1. **Author the `[JOURNEY]` case FIRST, from the model's value chain** (`/qa-test` Step 1e Part 0 →
`/qa-test-design` `test-design-techniques.md` §1a). It traverses the WHOLE chain in one run — trigger →
effect → persisted state → the surface the customer sees it on → what it unlocks — through the UI a
customer actually uses, with data that makes every link's outcome decidable. Stamp it
`Technique:FLOW` and title it `[JOURNEY]`. **This, not a single screen's happy path, is the baseline
the rest of the suite refines.** A feature that changes state and ships no journey case has not been
tested, however many per-screen cases surround it: on Loyalty Missions the biggest storefront suite
(71 cases) placed zero orders, so nothing in it could observe whether a mission ever advanced.
Then generate **1** case for the primary success flow of each additional screen the journey passes
through — the baseline; do not generate variations of it.
2. **Apply test design techniques selectively** — only where there is a real risk of failure:
- Boundary values: only for inputs where off-by-one or threshold errors are plausible in the implementation (e.g., quantity limits, price tier thresholds, discount caps)
- Equivalence partitions: only when the system has genuinely distinct code paths per partition
- State transitions: for lifecycle features (orders, quotes) where wrong-state transitions are a known failure mode
- Error guessing: for known VC platform quirks (e.g., double-click submit, GraphQL HTTP 200 ≠ success)
3. **Add negative cases** — at minimum 1 per happy path: the most likely real-world failure (invalid input, expired token, missing required field, unauthorized role). Do not generate negative cases for every possible invalid input — pick the one most likely to slip through.
4. **Add edge cases** — only from `ECL-*` patterns with documented failure history for this domain. Do not add edge cases speculatively.
5. **Add cross-domain cases** — only when the interaction point between domains is a known source of bugs (e.g., cart + coupon discount stacking, checkout + inventory reservation race).
6. **Cull before finalizing** — review the full candidate list and remove any case that: (a) duplicates the failure hypothesis of another case, (b) tests infrastructure rather than logic, (c) would only fail if the framework itself is broken, or (d) **names no chain link** — it neither crosses a link of the value chain nor guards one the journey already crosses. (d) is the cull that catches the expensive kind of waste: a strongly-asserted, well-formed check on something nobody's money depends on. `npm run tc:rank` cannot see it — it scores assertion *strength*, and it scored the missions storefront suite as strong while 54 of its 71 cases never left one page.
**A KEEP must survive three questions, and "it guards the UI" answers none of them:** name the *observable* it reads, the *defect* it would catch stated as a customer-visible failure, and *why that defect is plausible here* — a mechanism in this code, a bug in `reports/bugs/**`, or a `vc-bug-catalog` entry. "Any element could fail to render" is the null hypothesis and justifies infinite cases.
**Never invert an assertion to match a known defect.** A case whose correct expectation is currently unmet keeps the spec-derived expectation and links the bug (held at `Draft`, never promoted), or is marked `Manual`/`Deprecated` with the reason, or is cut so the finding lives only in the bug report. Flipping it to assert the broken behaviour makes the case green today, unfalsifiable forever, and RED on the day the bug is fixed.
### Step 3.5: VC-Specific State Patterns (check on every applicable feature)
When the feature contains any of the following field types, generate targeted cases for them. Each pattern has known failure modes in VC — these are not speculative.
#### On/Off Feature Flags & Active/Inactive Toggles
Appears in: store settings, promotions, coupons, catalog items, price lists, content pages, B2B org settings.
| State Scenario | Bug Hypothesis | Priority |
|---------------|---------------|---------|
| Item disabled → storefront must NOT show it or return it via API/GraphQL | Flag respected at display/query layer — common to forget filtering in one layer | P0 |
| Item disabled → API/GraphQL returns nothing, NOT a 5xx or empty data with errors | Disabled items often return 500 or malformed response instead of empty result | P0 |
| Toggle Off → save → reload page → still Off | Toggle state not persisted correctly to backend | P1 |
| Toggle On → Off → feature immediately unavailable | Cache invalidation on flag change — stale cache serves feature after deactivation | P1 |
| Inactive item in active category: category still visible, item filtered out | Cascade logic — parent active, child inactive, boundary filtering | P1 |
**Minimum cases:** 1 "disabled = not visible" case + 1 "state persists after save" case per toggle.
#### Start Date / End Date Fields
Appears in: promotions, coupons, price lists, content slots, scheduled imports.
| State Scenario | Bug Hypothesis | Priority |
|---------------|---------------|---------|
| Start date in future → entity NOT active yet (storefront/API must not apply it) | Date comparison uses server time vs client time; off-by-timezone errors | P0 |
| Start date past, end date future → entity IS active | The normal valid state — confirm it works, used as baseline | P0 |
| End date in past → entity expired → no longer applied | Expiry check at query time vs cached at creation time | P0 |
| Start date = today (boundary) → entity active from today | Boundary: inclusive vs exclusive comparison (>= vs >) | P1 |
| End date = today → entity still active today or already expired? | Boundary: end of day vs start of day; timezone offset issues | P1 |
| Start date after end date → validation error, entity not saved | Input validation — should reject before persistence, not silently swap values | P1 |
| No end date (open-ended) → entity active indefinitely | Null end date should not be treated as "expired at epoch" | P1 |
| Date range valid but entity also has active=false flag → inactive wins | Flag + date range interaction: both conditions must hold | P1 |
**Minimum cases:** expired case + future start case + boundary (today) case. Skip the no-end-date case only if the UI does not expose that option.
#### State Transitions (lifecycle objects)
Applies to: orders, quotes, RFQs, returns, import/export jobs.
Generate cases only for transitions that have business consequences:
- Valid transition: state A → state B → expected behavior/data change
- Invalid transition: state A → state C (should be rejected, not silently ignored)
- Side effects: does the transition trigger notifications, webhooks, index updates?
Do NOT generate a case for every possible transition — only those where rejection of an invalid transition or a missed side effect would be a real bug.
---
### Step 3.7: Prepare test-data combinations first (data-dependent cases)
Before populating any `Test_Data` column, **delegate combination design to
[`/qa-generate-data <feature>`](../qa-generate-data/SKILL.md)** — it learns the live variant
space, builds the pairwise matrix, reuses existing fixtures, authors only the gaps, and wires one
`@td()` **combination alias per Combo ID**. This makes the *source* of every `@td()` value explicit:
the prepared combinations, not invented data. Then **map one case (or case group) per Combo ID** so the
matrix and the suite stay traceable. The combination matrix it returns is your input to Step 4's
`Test_Data` column. Skip only for cases that touch no seeded entities (pure UI/copy/validation).
This composes with — does not replace — the no-hardcode rule enforced in Step 5.
### Step 3.9: Scaffold the rows — do not hand-type the boilerplate
**The three KEEP questions in step 6 above are now machine-checked, one step earlier.** Write the surviving
candidates out as an **authoring plan JSON** (one per target suite) and run:
```bash
npm run tc:alloc -- --prefix <PREFIX> --block <layer>=<n> [--block ...] # once, before any fan-out
npm run tc:scaffold -- --plan <plan>.json --id-block <PREFIX-NNN..PREFIX-NNN> --out <staged>.csv
```
`scripts/test-cases/scaffold-rows.ts` **rejects a planned row that cannot name** its `observable` (the
value it reads), its `defect` (the failure a CUSTOMER would see — null-hypothesis phrasings like *"could
fail to render"* are refused by name), and its `plausible` (one of the three grounds step 6 lists: a
`VC-*` catalog entry, a filed bug, or `mechanism: <what in this code makes it likely>`). A row that
cannot justify itself never becomes a CSV row, so it is never authored, reviewed, executed or maintained
— which is the cull step 6 asks for, moved to where it is cheap.
What it emits:
- A **staged CSV** (canonical header, in the scratchpad — never `regression/suites/`) with ten columns
already derived from `(layer, priority, archetype, technique, ticket)`: ID, Section, Priority,
Business_Rule, Edge_Case_Refs, Test_Data, Cross_Layer_Checks, Failure_Signals, Cleanup, References,
Automation_Status=`Draft`. Only **Preconditions / Steps / Assertions** are left blank — Step 4 below
authors exactly those, and `npm run suites:review -- <staged>.csv` names each unfilled one.
- A **`.design.md` sidecar** — the per-row KEEP answers, i.e. the record of *why each case exists*.
**Sweeps expand mechanically**, with their bug hypothesis already written by the document that owns them
(read at run time, never transcribed): `state-stress` (qa-design §State-Stress Pass) · `uip`
(modern-web-attack-surface §`UIP-*`) · `toggle` and `date-range` (§3.5 above). Declare them in the plan's
`sweeps[]` with the surface being swept; waiving an item needs a reason, because a silent omission is
what makes a sweep unreportable. This is the cheapest coverage in the pipeline and it is where the corpus
is measurably thinnest (8 · 5 · 11 · 5 · 5 `UIP-*` cases across 1,961 Frontend cases).
`--id-block` comes from `tc:alloc`, which scans the corpus **once**; the scaffolder refuses to spill past
the block it was given. That is what makes concurrent per-surface batches safe — see `/qa-test` Step 3b.
### Step 4: Write Each Test Case
Fill `Preconditions` / `Steps` / `Assertions` on the scaffolded rows. The remaining columns are already
populated by Step 3.9 — do not retype them; if one is wrong, fix the plan and re-scaffold, so the plan
stays the source of truth. When scaffolding is skipped (a one-off single row), populate all 15 columns
following the template:
```
ID, Title, Section, Priority, Business_Rule, Edge_Case_Refs, Preconditions, Test_Data, Steps, Assertions, Cross_Layer_Checks, Failure_Signals, Cleanup, References, Automation_Status
```
**Column-by-column checklist:**
- [ ] **ID** — `PREFIX-NNN` format, sequential, never reuse. Prefix matches suite (e.g., `SMK`, `CART`, `PAY`, `AUTH`, `API`)
- [ ] **Title** — `[Subject] — [Action/Scenario]` pattern, short and action-oriented
- [ ] **Section** — `Suite > Domain > Sub-area` hierarchy
- [ ] **Priority** — Critical/High/Medium/Low based on risk and business impact
- [ ] **Business_Rule** — At least one `BL-*` ID (leave blank only for pure UI tests)
- [ ] **Edge_Case_Refs** — `ECL-*` IDs if the case covers a known edge case pattern
- [ ] **Preconditions** — Human-readable state requirements, use `{{VAR}}` for env values. Express as **state**, never as "after running <ID>" (ISTQB independence rule). If setup duplicates another case's first ≥70% of steps, use `Preconditions: state from <ID> (state summary)` instead of restating the flow (avoid-repetition rule).
- [ ] **Test_Data** — Only `key={{VAR}}` bindings, comma-separated
- [ ] **Steps** — Every step tagged: `[NAV]`, `[ACT]`, `[WAIT]`, `[SCROLL]`, `[KEY]`. One action per line. WAIT after every state-changing ACT
- [ ] **Assertions** — Tagged: `[DOM]`, `[STATE]`, `[MATH]`, `[FORMAT]`, `[NAV]`. Explicit predicates, no vague language. **≥1 assertion of class `INV`/`REL`/`DER`/`SHAPE`** — a case whose assertions are all `PRES` (visible / shown / present / renders) is rejected by `T-006`. For a Storefront layout/geometry case prefer the measurable tags (`[SHIFT]` `[TOUCH]` `[SPACING]` `[ALIGN]` `[OVERFLOW]` `[CLS]`) over prose inside `[DOM]`. See `test-case-template.md` §Assertions + §Measurable UI vocabulary
- [ ] **Cross_Layer_Checks** — Tagged: `[API]`, `[CONSOLE]`, `[NETWORK]`, `[ADMIN]`, `[EMAIL]`. Every mutation MUST check `errors[]` is empty
- [ ] **Failure_Signals** — At least 2: one timeout signal + one API/console signal
- [ ] **Cleanup** — State restoration or `none`
- [ ] **References** — **REQUIRED for Critical/High:** JIRA ticket (`VCST-XXXX`), `REQ-*` ID, or user-story link. `BL-*` IDs alone do NOT satisfy this — those belong in `Business_Rule`. Infrastructure/smoke cases use `smoke-baseline` placeholder (never empty).
- [ ] **Automation_Status** — `Draft` for just-generated cases (default out of this skill). Promote to `Reviewed` only after `/qa-review-tests` returns ≥ PASS WITH WARNINGS AND a human/`qa-lead-orchestrator` approves. `Automated`/`Manual`/`Semi-Automated` = execution mode (implies Reviewed).
### Step 4.5: Provenance tagging (ground every assertion)
Generation is offline, so you tag by best-available source. For **each assertion line**, append a
`{...}` provenance tag (grammar in `test-case-template.md` → Assertions column):
1. **`{SPEC}`** — the expected behavior is stated in the **tracker ticket's** requirement/AC (Jira or
Azure Boards). This is the workhorse for a new feature.
2. **`{BL}`** — it restates a real `BL-*`/`BL-UI-*` invariant from `business-logic.md`.
3. **`{DOC}`** — you confirmed it in VirtoOZ docs or product source (`/vc-docs`,
`PlatformFrontendSourceCode`, an i18n file). Only tag `{DOC}` if you actually looked it up.
4. **`{HYPOTHESIS}`** — none of the above; it is an educated guess of a plausible bug. **Phrase it as a
question** ("verify whether…"), never as a fact. Do NOT invent literal message strings on these lines
(assert the semantic — literal-text rule).
Do **not** emit `{OBSERVED}` during generation — that class is reserved for the live `--verify` pass,
which is the only step that may confirm a behavior against the deployed build. **New-feature path:**
offline you will have mostly `{SPEC}` + `{HYPOTHESIS}`; those must be live-verified (upgraded to
`{OBSERVED}`) by a mandatory `--verify` run before the suite can be promoted past `Draft`.
### Step 5: Validate & Output
1. **Self-review each case** against the writing guidelines in `test-case-template.md`:
- **Every assertion carries a provenance tag** (`{SPEC}`/`{BL}`/`{DOC}`/`{HYPOTHESIS}`); no untagged
lines. No literal message/validation strings on `{HYPOTHESIS}` or unconfirmed `{SPEC}` lines —
assert the semantic (literal-text rule, DV-016 twin)
- No assertions mixed into Steps
- No hardcoded URLs/emails/passwords (all `{{VAR}}`); no hardcoded entity-specific values — IDs, SKUs, prices, addresses, coupons, test cards, order numbers — all resolved via `@td(ALIAS.field)` against [`test-data/aliases.json`](../../../test-data/aliases.json) (see `../testing/qa-postman/test-data-fixtures.md`)
- Every mutation has `errors[]` check in Cross_Layer_Checks
- At least 2 failure signals per case
- **Layer-correct tags**: API cases use `[HTTP]`/`[STATUS]`/`[BODY]`, not `[NAV]`/`[ACT]`
- **GraphQL cases** always include `[ERRORS] errors[] is empty` assertion
- **GraphQL cases** validated against `graphql-schema.md`: query/mutation names, arg names, command wrapper, response field names, MoneyType structure (Step 2.5 checklist)
- **Admin cases** use `[BLADE]`/`[GRID]`/`[SAVE]` tags, not generic `[ACT]` for blade interactions
- **E2E cases** have `--- LAYER ---` markers and assertions from ≥2 layers
2. **Check for duplicates** against existing suite cases
3. **Output format** — present as a **Feature Test Matrix** grouped by layer:
```
## Feature Test Matrix: [Feature Name] (VCST-XXXX)
### Layer: REST API (N cases → Suite 14)
| ID | Title | Priority | Business Rules | Technique |
[CSV block for API cases]
### Layer: GraphQL xAPI (N cases → Suite 15)
| ID | Title | Priority | Business Rules | Technique |
[CSV block for GraphQL cases]
### Layer: Admin UI (N cases → Suite NN)
| ID | Title | Priority | Business Rules | Technique |
[CSV block for Admin cases]
### Layer: E2E Cross-Layer (N cases → Suite 00/NN)
| ID | Title | Priority | Business Rules | Technique |
[CSV block for E2E cases]
### Coverage Summary
| Layer | P0 | P1 | P2 | Total | BL-* Coverage |
| Traceability: VCST-XXXX → [all generated case IDs]
```
4. **Suggest placement** — which suite file each layer's cases should be added to
5. **Cross-layer traceability** — every case in every layer links back to the same feature (VCST-XXXX in References column). The Feature Test Matrix header connects them all.
## Modes
### Generate Mode (default)
Produce new test cases and present them for review. Do not modify suite files without explicit confirmation.
### Migrate Mode (`migrate NN`)
Transform legacy TestRail-format CSV rows to the enriched format:
1. Read the existing suite CSV
2. For each row, apply the migration mapping from `test-case-template.md` (Migration section)
3. Split `Steps` → `Steps` + `Assertions`
4. Split `Expected Result` → `Assertions` + `Cross_Layer_Checks`
5. Split `Preconditions` → `Preconditions` + `Test_Data`
6. Add new columns: `Business_Rule`, `Edge_Case_Refs`, `Failure_Signals`, `Cleanup`
7. Remove: `Type`, `Estimate`
8. Present the migrated CSV for review
### Extend Mode (`suite NN`)
Read existing suite, analyze gaps, generate cases to fill them:
1. Read the current suite CSV and count cases per section
2. Cross-reference with the domain checklist(s) for that suite
3. Identify uncovered checklist items or missing edge cases
4. Generate new cases only for the gaps
5. Use the next available ID in the suite's numbering sequence
## Output Example
```csv
PAY-042,"CyberSource — Expired Card Rejection","Payment > CyberSource > Validation",High,BL-PAY-001,"ECL-1.1, ECL-1.3","User logged in, cart has items, CyberSource payment selected on cart page","email={{USER_EMAIL}}, password={{USER_PASSWORD}}, front_url={{FRONT_URL}}","[NAV] {{FRONT_URL}}/cart
[WAIT] cart loaded with items
[WAIT] CyberSource payment form iframe visible
[ACT] fill card number: 4111111111111111
[ACT] fill expiry: 01/23
[ACT] fill CVV: 123
[ACT] click 'Place Order'
[WAIT] form validation response","[DOM] error message displayed indicating expired card
[DOM] 'Place Order' button re-enabled after error
[STATE] order NOT created — cart still intact","[API] payment authorization returns decline code
[CONSOLE] no unhandled JS errors
[NETWORK] no 5xx responses from payment gateway","Payment spinner visible >10s, 5xx from payment endpoint, console TypeError, blank payment form","none — order was not placed",VCST-4648,Automated
```
## Layer → Agent Delegation
Generated test cases route to the correct executing agent by layer:
| Layer | Execute With | Browser | Suite Target |
|-------|-------------|---------|-------------|
| REST API | `qa-backend-expert` | `playwright-edge` or Postman MCP | `Backend/api/049-*.csv` |
| GraphQL xAPI | `qa-backend-expert` | `playwright-edge` or Postman MCP | `Backend/graphql/050*.csv` |
| Admin UI | `qa-backend-expert` | `playwright-edge` or Chrome DevTools | `Backend/<module>/*.csv` (by module) |
| Storefront UI | `qa-frontend-expert` | `playwright-chrome` | `Frontend/<area>/*.csv` (by area) |
| E2E Cross-Layer | `qa-frontend-expert` + `qa-backend-expert` | coordinated | Suite 00 or feature suite |
| Storybook/A11y | `ui-ux-expert` | Chrome DevTools | Separate |
## Integration with Other Skills
| Skill | Relationship |
|-------|-------------|
| `/qa-checklist` | Checklists are input — each item becomes 1-3 test cases |
| `/qa-test-design` | Techniques (EP, BVA, decision tables) drive case derivation |
| `/qa-generate-data` | **Prepare data first (Step 3.7)** — designs the cross-entity combinations + `@td()` combination aliases each data-dependent case references; map ≥1 case per Combo ID |
| `/qa-risk` | Risk level determines priority assignment and case count |
| `/qa-coverage-gap` | Gap analysis identifies where new cases are needed most |
| `/qa-plan` | Generated cases feed into test plans |
| `knowledge/domain/sitemap.md` | Sitemap provides URLs and navigation context for steps |
| `/qa-api ref` | xAPI reference for Cross_Layer_Checks assertions |
| `../testing/qa-postman/test-data-fixtures.md` | `@td(ALIAS.field)` resolver, [`test-data/aliases.json`](../../../test-data/aliases.json) registry, fixture conventions — read before populating `Test_Data` or `Preconditions` columns with any entity-specific value |
| `knowledge/api/graphql-test-cases-runner.md` | Runner-native CSV authoring contract — read before writing any GraphQL test case (Step 2.5 enforces this) |
| `knowledge/api/graphql-schema.md` | Live introspection snapshot — verify every query/mutation name and field against this (Step 2.5 enforces this) |
## Rules
- **Ground before you assert** — every assertion carries a provenance tag; anything not traceable to `{SPEC}`/`{BL}`/`{DOC}` is a `{HYPOTHESIS}` phrased as a question, and no `{HYPOTHESIS}`/untagged case reaches `Reviewed`. For a new feature (no doc/source), the mandatory `--verify` live pass upgrades hypotheses to `{OBSERVED}`. Never invent literal message strings — assert the semantic (literal-text rule).
- **Bug hypothesis first** — every case must answer: "what real bug does this catch?" If you cannot answer, do not generate the case. Coverage numbers are vanity metrics.
- **Chain link, or it does not ship** — every case either **crosses** a link of the feature's value chain or **guards** one the `[JOURNEY]` case already crosses; a case that does neither is decoration and is culled (Step 3 §6d). A link is crossed only by an observation on the *far side* of it: reading a value from the API and reading the same value off the page are two observations of one link, never coverage of the join between them.
- **One `[JOURNEY]` case per state-changing feature, authored first** (Step 3 §1), `Technique:FLOW`. It is the case that answers "does this feature work at all?" — the one everybody assumes someone else wrote.
- **A filtered read must be PROVED to filter.** Run the query once WITHOUT the filter and confirm the
results differ before any conclusion rests on the filtered figure. An ignored parameter does not error
— it returns a plausible number for a wider population, and a comparison between two such numbers is a
comparison of the whole dataset with itself. Measured 2026-09-01 on
`POST /api/loyalty-program-operation-log/search`: `{userIds:[uid]}` (plural) and an **unfiltered**
query return byte-identical results, `totalCount 990` across 4 distinct users, while `{userId: uid}`
(singular) returns `19` for one — so a per-user claim was being made from a global count. This is the
same "check the instrument against a known-good control" move that catches a wrong auth context,
applied to a query parameter; see also `.claude/knowledge/api/graphql-schema.md` on optional arguments
that are accepted and answer for a context you never chose.
- **A verification that reports on a RECORDED SNAPSHOT is not checking the thing it names.** Same
family as the one below, one step removed: the check does not share the action's implementation, it
shares the action's *memory*. Measured 2026-09-01: `td:validate:missions-e2e` reported the fixture
set **clean** while three of its five fixtures were consumed — two terminal `Completed`, one
half-spent — because it validates the seed-time baselines recorded in the overlay rather than the
current state of the entities. It therefore certifies a fixture set that cannot run, and it does so
with a green exit code. A guard over live state must READ live state; a guard over a recorded
baseline is a guard over the record, and must say so in its own output.
- **A verification that shares its implementation with the thing it verifies cannot detect that
implementation's failure — and it fails CLEAN.** This is the strongest form of false pass we have
measured, because the check passes *because* the action failed the same way. Worked example
(2026-09-01): a teardown reported `deleted 3 fixture product(s) ✓ zero residue verified` while a
fourth product survived — the residue check called the same lookup helper the delete had used, and
that helper's window had truncated identically both times. Not a narrower set: **the same blind spot
asked twice.** So a verification step must reach the state by a *different path* than the action did
— a different endpoint, a different index, a different reader — or it is a restatement of the
action's own belief about itself.
- **A clean negative is the most dangerous result in the suite — prove the mechanism fired.** An
assertion of the form "X did not happen" passes identically when X correctly did not happen and when
nothing ran at all, or when the field you read cannot express X in the first place. Three instances
measured on 2026-08-28, all of which read as a confident PASS: a `0%` progress card that is
byte-identical whether nothing accrued or the settlement job never started (the API fabricates a
transient zero in memory); an assertion that "no ledger row references this order" on a subsystem
whose mission rows carry `object === null`, so **no** row ever references an order and the check
could not fail; and a role's credentials read as empty because they were resolved off a curated
export that omits the key, which returns `undefined` exactly like a genuinely unset variable. Every
negative assertion therefore needs a **positive control in the same event** — a second observation
that MUST move if the mechanism ran — and *neither moving is not a pass*.
- **When the attribution ceiling drops, assertions that depended on the higher one go VACUOUS, not
red.** If you discover that a field you were pinning on does not exist, is always null, or means
something else, re-audit every assertion that leaned on it: they will keep passing, which is why
nobody notices. Re-check the whole file, not the one case that prompted the discovery.
- **No case may certify a defect** — the expected result comes from the specification, never from the current build (Step 3 §6). A suite tuned to the implementation is unfalsifiable by construction and inverts the day the bug is fixed.
- **Minimum effective set** — a smaller suite of targeted cases is better than a large suite of shallow ones. Prefer 5 focused cases over 20 that repeat the same failure mode.
- **Suite sizing & packing (one suite = one runnable unit)** — pack cases so each CSV is a single feature/module area that **one isolated agent reads and runs in one session** (`/qa-regression` dispatches exactly one QA-expert agent per CSV, batched 3 at a time — the browser pool):
- **Target ~20–40 cases per CSV** (repo median is 28). ≤20 is fine for a small feature; treat **>40 as a signal to split**, not a target to fill.
- **Split by feature with a suffix**, never by growing one file: `040a/040b/040c` (payment processors), `050b1–050b5` (xCart), `072/072b/072c/072d` (configurable products). Each split suite stays scoped to one runner.
- **Cap expensive browser-driven suites at ≤8 cases** — long runner sessions (>2h) produce unreliable results; a suite must finish inside its CI turn/time budget (`MAX_TURNS` default 100, ~10 min per suite).
- **Never pad to hit a number** — cull duplicates first (Step 3 §6), then split only when the *legitimate* case count outgrows one session. Smoke aggregators (042/078) are deliberate exceptions.
- **Counts live only in `config/test-suites.json`** (`testCount` per suite) — never restate them in the CSV, the suites README, or here. Regenerate/verify with `npm run suites:sync` / `npm run suites:lint`.
- **Format is non-negotiable** — every case MUST use all 15 columns from `test-case-template.md`
- **No vague assertions** — "page loads correctly" is not an assertion.
- **No presence-only cases.** Every case needs ≥1 assertion of class `INV`/`REL`/`DER`/`SHAPE`
(`test-case-template.md` §Assertions). `[DOM] product title visible` is class `PRES` — legal
as a *guard*, never as the case's only check: it passes when the title renders the wrong product.
The strong forms are `[REL] PDP title == listing title for the same SKU`,
`[DOM] title equals @td(PROD_CFG_BIKE.name)`, or `[SHIFT] topDelta == 0` for a layout case.
- **No compound steps** — one action per `[ACT]` line. Wrong: `click Add to Cart and verify badge`. Right: separate `[ACT]` and `[ASSERT]`. This rule still applies inside journey cases — each action is one `[ACT]`; the journey is built from many sequential `[ACT]`/`[ASSERT]` rounds, not from compound lines
- **Frontend journeys are exceptions to atomicity** — for Storefront UI flows where behavior depends on cross-screen state (checkout, cart→order, login+purchase, BOPIS end-to-end, address/org switch mid-journey), write one journey case with `--- SCREEN: <name> ---` dividers in `Steps`, `[JOURNEY]` tag in `Section`, and an `[ASSERT]` at every screen boundary. Do NOT shard into atomic per-screen cases. See `agents/test-management-specialist.md` → Frontend Journey Exception for full criteria
- **Always resolve test data, never hardcode** — URLs and credentials use `{{VAR}}` (env-backed). Entity-specific values — IDs, SKUs, prices, emails, addresses, coupon codes, test-card numbers, order numbers, virtual-catalog roots, URL path segments — use `@td(ALIAS.field)` resolved against [`test-data/aliases.json`](../../../test-data/aliases.json). Hardcoded fixtures rot when catalogs are reseeded or orgs are recreated. See `../testing/qa-postman/test-data-fixtures.md` for the full resolver contract and patterns. Validate with `npx tsx scripts/test-data/validate-td-refs.ts`
- **Every mutation → `errors[]` check** — GraphQL HTTP 200 does not mean success in xAPI
- **Minimum 2 failure signals** per case (timeout + API/console)
- **Negative cases are mandatory** — for every happy path, generate at least one negative/error case — pick the failure mode most likely to slip through, not all possible invalid inputs
- **Per-feature P+N+B mix (ISTQB)** — any feature group with ≥3 cases MUST include at least 1 positive + 1 negative + 1 boundary case (boundary waived only if the feature has no ordered/numeric input). Verified by `/qa-review-tests` Dimension 9 (TC-001)
- **Cases must be independent (ISTQB)** — `Preconditions` express required **state**, never "after running <ID>". Order-dependence between cases is forbidden; use `state from <ID>` to reference an earlier case's end-state by its described state, not by its execution
- **Avoid repetition by reference** — if two cases share ≥70% of setup steps, the later case uses `Preconditions: state from <ID> (summary)` and `Steps:` starts from the point of divergence. Do NOT restate login/navigate/add-to-cart flows across cases in the same suite
- **Requirement traceability is mandatory for Critical/High** — `References` column MUST contain a JIRA ticket or `REQ-*` ID for any Critical/High case. `BL-*` alone is not traceability — it is business-rule mapping
- **Cases leave this skill as `Draft`** — peer review via `/qa-review-tests` + human approval promotes to `Reviewed`. Only `Reviewed`+ cases enter regression selections
- **ID stability** — never reuse or renumber IDs. Deleted cases leave gaps in numbering
- **Ask before writing** — present generated cases for review before appending to any suite CSV file
- **Append via the safe writer, never hand-rolled** — once approved, append with
`npm run suites:append -- <target-suite.csv> --rows <new-rows.csv>` (`scripts/test-cases/append-test-cases-to-suite.ts`).
It validates the 15-column schema, escapes commas/newlines in Steps/Assertions, guarantees the boundary
newline, dedup-checks by ID + Title+Section, and round-trip-verifies the append (the corruption from a
hand-rolled `appendFileSync` — merged 29-field rows — is what prompted this). Use `--dry-run` to preview.
Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.
No comments yet. Be the first to comment!