Qa Coverage Gap

---
name: qa-coverage-gap
description: "[Testing] Autonomous test coverage gap analysis and generation — identifies missing test cases, generates enriched CSV test cases, validates P0 cases via browser, and reports improvements."
argument-hint: "analyze | generate | validate | full | domain <name> | suite <ID>"
disable-model-invocation: true

---

# /qa-coverage-gap — Test Coverage Gap Analysis & Generation

Autonomously improves test coverage by identifying gaps between application features and existing test suites, generating new test cases in the enriched agent-native format, validating P0 cases, and reporting improvements. Operates in a 4-cycle iterative pipeline.

For sprint- or release-level multi-domain runs, run this skill once per domain (the orchestrated `/qa-coverage-generation` twin was removed 2026-09-08 with zero recorded runs).

## Usage

```
/qa-coverage-gap                    # Run full 4-cycle pipeline (analyze → generate → validate → report)
/qa-coverage-gap analyze            # Cycle 1 only: gap analysis with prioritized report
/qa-coverage-gap generate           # Cycles 1-2: analyze + generate test cases
/qa-coverage-gap validate           # Cycles 1-3: analyze + generate + validate P0 cases
/qa-coverage-gap full               # All 4 cycles including commit
/qa-coverage-gap domain <name>      # Focus on a manifest domain (purchase-flow, catalog-search, …)
/qa-coverage-gap suite <ID>         # Focus on a specific suite by ID (e.g., 028, 050b1, 072c)
```

`<name>` for `domain` accepts any manifest domain from `config/test-suites.json` `_meta.domains`: `purchase-flow`, `catalog-search`, `auth-security`, `marketing`, `content-cms`, `customer-b2b`, `platform-config`, `communication`, `branding`, `cross-cutting`.

## Supporting Files

- **coverage-gap-methodology.md** — Gap detection heuristics, TestRail cross-reference rules, priority scoring matrix, gap categories
- **feature-domain-map.md** — Manifest-domain inventory with expected coverage thresholds

## Architecture

### Cycle 1 — Analysis (always runs)

1. **Read regression coverage** — every suite CSV referenced in `config/test-suites.json` (`suites[*].file`). Use the manifest's `domain` / `layer` / `concern` / `priority` fields for routing — never hardcode suite IDs.
2. **Cross-reference TestRail baselines** in `test-suites ( export from Test-rail )/` — surface `MIGRATION_GAP` and `SHALLOW_MIGRATION` per `coverage-gap-methodology.md` §1b.
3. **Read feature inventory** from:
   - `knowledge/oracles/business-logic.md` — `BL-*` invariants
   - `knowledge/oracles/e-commerce-edge-cases-library.md` — `ECL-*` edge cases
   - `knowledge/domain/sitemap.md` — storefront page inventory
   - `knowledge/execution/module-suite-map.md` — module-to-suite mapping
   - `knowledge/api/graphql-schema.md` — xAPI schema reference (REQUIRED before authoring any GraphQL case)
   - `knowledge/domain/products.md`, `catalog.md`, `store-settings.md` — realistic test-data context
   - `skills/qa-plan/e2e-scenario-catalog.md` — 105 E2E scenarios
   - `skills/qa-checklist/domain-checklists.md` — UI/UX storefront checklists
   - `skills/qa-checklist/backend-admin-checklists.md` — Admin SPA & module checklists
   - `skills/qa-checklist/graphql-checklist.md` — GraphQL-specific checklist
   - `skills/qa-api/xapi-query-ref.md` — endpoint inventory
   - `skills/qa-api/test-cases-api-graphql.md` — API/GraphQL test-case patterns
4. **Live VC documentation (Context7)** — for the in-scope domain(s):
   - `mcp__context7__resolve-library-id { libraryName: "virtocommerce" }` → `/virtocommerce/vc-docs`
   - `mcp__context7__query-docs { libraryId, query, tokens: 8000 }` — query stems per `.claude/templates/agent-dispatch.md` § Sample Queries by Domain
   - Flag features documented in VC docs but missing from current coverage.
5. **Map every test case** to its manifest domain via the manifest's `domain` field; supplement with `Section` column and title keywords only when the manifest is ambiguous.
6. **Identify gaps** per `coverage-gap-methodology.md` categories (`ZERO_COVERAGE`, `SHALLOW_HAPPY`, `MISSING_NEGATIVE`, `MISSING_INTEGRATION`, `MISSING_CROSS_DOMAIN`, `STALE_COVERAGE`, `MIGRATION_GAP`, `SHALLOW_MIGRATION`).
7. **Score gaps** using the 4-factor matrix in `coverage-gap-methodology.md` §3:
   - Revenue impact (40%) · User frequency (25%) · Failure severity (20%) · Existing coverage (15%)
   - 8.0–10.0 → P0 · 5.0–7.9 → P1 · 2.0–4.9 → P2 · <2.0 → P3 (excluded from generation)
8. **Output:**
   - `reports/coverage/{RUN_ID}/gap-inventory.json` (one typed record per gap; schema in `coverage-gap-methodology.md` §`gap-inventory.json`)
   - `reports/coverage/{RUN_ID}/gap-analysis.md` (digest)

Definition of Done: every gap has `manifestDomain`, `applicableLayers[]`, `targetSuites[]` resolved against the manifest, `priorityScore`, `priority`, and `gapCategory`.

### Cycle 2 — Generation (runs with `generate`, `validate`, `full`)

1. For each gap, invoke `/qa-test-cases-generator --layer <csv-list>` once per applicable layer. Layers: `api`, `graphql`, `admin`, `storefront`, `e2e`.
2. Format contract: `skills/qa-test-cases-generator/test-case-template.md` (15-column enriched CSV: ID, Title, Section, Priority, Business_Rule, Edge_Case_Refs, Preconditions, Test_Data, Steps, Assertions, Cross_Layer_Checks, Failure_Signals, Cleanup, References, Automation_Status).
3. **For Backend/graphql/* suites (`050a`–`050k`):** authoring contract is `knowledge/api/graphql-test-cases-runner.md` — runner-native tags (`[AUTH]/[GQL-OP]/[GQL-VARS]/[GQL-EXEC]/[GQL-CAPTURE]/[REST-OP/EXEC/CAPTURE]/[ERRORS]/[DATA]/[NULL]/[COUNT]/[VAR]`). Browser-mode `[GQL]` tags are **not** valid in these suites.
4. **Test-data contract (mandatory)** per `.claude/rules/test-data.md` — resolve via `{{VAR}}`, `@td()`, `live-discover`, or `random-data`. Literal IDs/SKUs/emails/prices/order-numbers/paths are review failures. Use `AGENT-TEST-` prefix for generated entities so `/qa-seed-data teardown` reclaims them.
5. **Deduplication** — before appending, read target suite CSV and skip semantic duplicates (matching `Title + Section` OR `Steps + Assertions`).
6. **Test-case quality rules:**
   - Unique ID with domain prefix (next sequential within the suite)
   - Deterministic numbered steps with typed tags
   - Explicit assertions (predicate-driven, no "verify it looks correct")
   - Happy path → critical error paths → edge cases (boundary, negative, cross-domain)
   - Priority mapping: P0 = Critical, P1 = High, P2 = Medium
7. **Append** cases to the target suite CSVs resolved in Cycle 1 **via the safe writer** —
   `npm run suites:append -- <target-suite.csv> --rows <new-rows.csv>` (`scripts/test-cases/append-test-cases-to-suite.ts`).
   Never hand-roll the append: the writer enforces the 15-column schema, escapes commas/newlines, guarantees
   the boundary newline, dedup-checks (so step 5's dedup is also enforced in code), and round-trip-verifies.
   Use `--dry-run` in Cycle 4's quality gate to confirm a clean append before committing.
8. If no existing suite matches, mark `blocked:needs-suite` and surface in the report — never auto-create suite files.

Definition of Done: every generated case conforms to the template, references at least one `BL-*` invariant (or `ECL-*`), and `npx tsx scripts/test-data/validate-td-refs.ts` passes against all modified CSVs.

### Cycle 3 — Validation (runs with `validate`, `full`)

1. Execute each new **P0 case** against QA via the assigned browser (default: `playwright-chrome`; fallback per `defaults.fallbackChain` in `config/test-suites.json`).
2. If steps don't work (element not found, flow changed), revise the case once; if still broken, mark `Automation_Status = needs-review` with a Failure_Signal note.
3. Mark `Automation_Status = validated | needs-review | pending` per outcome.
4. Cleanup created test data using the `AGENT-TEST-` prefix (see `knowledge/execution/live-discovery.md` § Cleanup).

Definition of Done: ≥80% of new P0 cases reach `validated`; remainder are flagged `needs-review` with explicit failure signals (not silently skipped).

### Cycle 4 — Report & Commit (runs with `full` only)

1. **Quality gate** — must all pass before commit:
   - Generated cases conform to `test-case-template.md`
   - `validate-td-refs` passes
   - If any modified suite is in critical-UI scope: `npm run scope:validate` exits 0
   - P0 validation pass rate ≥ 80%
2. Update `reports/coverage/{RUN_ID}/coverage-generation-report.md` with before/after comparison, batch results, TestRail migration reconciliation, Context7 findings → cases, remaining gaps.
3. Update `config/test-suites.json` `testCount` per modified suite (or invoke `npm run suites:sync` if available).
4. Git commit message format: `test(coverage): add N new test cases covering [areas] - automated gap analysis (RUN_ID)`. Do not push.

If quality gate fails, abort Cycle 4 — leave CSVs as written (user reviews diff before manual commit).

## Manifest-Domain Routing

This skill routes by the manifest domain (`config/test-suites.json` `suites[*].domain`), not by 2-digit suite IDs.

| Manifest Domain | Typical Layers | Lead Sub-agent |
|----------------|----------------|----------------|
| `purchase-flow` | storefront, graphql, e2e | `qa-frontend-expert` |
| `marketing` | storefront, admin, e2e | `qa-frontend-expert` |
| `auth-security` | api, admin, e2e | `qa-backend-expert` |
| `customer-b2b` | api, graphql, admin, e2e | `qa-backend-expert` |
| `communication` | api, admin, e2e | `qa-backend-expert` |
| `catalog-search` | api, graphql, admin, storefront | `qa-testing-expert` |
| `platform-config` | api, graphql, admin | `qa-testing-expert` |
| `content-cms` | admin, storefront | `qa-testing-expert` |
| `branding` | admin, storefront | `qa-testing-expert` |
| `cross-cutting` | storefront, e2e | `qa-testing-expert` |

**Target-suite resolution rule** (run for every gap):

```text
targetSuites = suites where
  suite.domain === gap.manifestDomain
  AND suite.layer matches the layer being generated (frontend|backend)
  AND suite.concern is appropriate for the layer:
      api / graphql → concern: "api"
      admin         → concern: "admin"
      storefront    → concern: "functional" and layer: "frontend"
      e2e           → concern: "functional" (cross-domain)
```

Never hardcode suite IDs in this skill — query the manifest. The manifest currently has 118 suites including sub-suites such as `050a`–`050k`, `072b`–`072d`.

## Integration with Other Skills & Commands

| Direction | Skill / Command | Relationship |
|-----------|-----------------|--------------|
| Upstream | `/qa-plan` | E2E scenario catalog provides expected coverage |
| Upstream | `/qa-checklist` | Domain checklists define expected test areas |
| Upstream | `/qa-api ref` | API/GraphQL reference inventory |
| Upstream | `/qa-risk` | 5×5 matrix as tie-breaker for priority scores within ±0.5 |
| Upstream | `knowledge/domain/sitemap.md` | Page inventory |
| Upstream | `knowledge/api/graphql-schema.md` | Schema verification before authoring GraphQL cases |
| Downstream | `/qa-test-cases-generator` | Receives `--layer` invocations to author cases |
| Downstream | `/qa-review-tests` | Reviews generated cases on 11 dimensions before merge |
| Downstream | `/qa-test` | Generated cases can be executed by qa-testing-expert |
| Downstream | `/qa-regression` | Updated suites feed into regression runs |
| Downstream | `/qa-metrics` | Coverage metrics updated post-generation |

## Rules

**Architecture:**
- Single-agent execution by design — for sprint/release-level scope, run once per domain.
- Resolve target suites by querying `config/test-suites.json` — never hardcode suite IDs in this skill.

**Format & data:**
- All generated cases follow `skills/qa-test-cases-generator/test-case-template.md` (15-column enriched CSV).
- Backend/graphql/* (`050a`–`050k`) cases follow the runner-native authoring contract in `knowledge/api/graphql-test-cases-runner.md`. Never use browser-mode `[GQL]` tags inside those suites.
- Test data is never hardcoded — `{{VAR}}` / `@td()` / `live-discover` / `random-data` per `.claude/rules/test-data.md`.
- `npx tsx scripts/test-data/validate-td-refs.ts` MUST pass before Cycle 3 begins (interactive) or before Cycle 4 commits (`full`).
- Use the `AGENT-TEST-` prefix for any new test data so `/qa-seed-data teardown` reclaims it.
- Generated steps must be MCP-executable (no manual-only steps).
- Never hardcode environment URLs — use `{FRONT_URL}` / `{BACK_URL}` patterns.

**Suite handling:**
- Never create new suite files — flag `blocked:needs-suite` and surface in the report.
- Preserve existing IDs — new cases get the next sequential ID in their domain prefix.
- Mark gaps requiring new test-data creation as `blocked:test-data`.

**Validation & gates:**
- P0 validation requires actual browser execution, not just review.
- Quality gate (Cycle 4): format conformance + `validate-td-refs` + `scope:validate` (when applicable) + ≥80% P0 pass rate — all must pass before commit.

**Knowledge sources:**
- Gap analysis must reference `business-logic.md` invariants (`BL-*`) for priority scoring and case authoring.
- Always query Context7 (`/virtocommerce/vc-docs`) during Cycle 1 — never rely solely on local knowledge files.
- For GraphQL cases, consult `graphql-schema.md` (live introspection) before authoring queries/mutations.

**Output:**
- All artifacts under `reports/coverage/{RUN_ID}/` — never repo root.