Test Loop

---
name: test-loop
description: "Plan, generate, and heal an executable E2E test suite from approved acceptance criteria (web and mobile)."
metadata:
  triggers:
    keywords:
    - test loop
    - workflow
---
# Test Loop Skill

> [!IMPORTANT]
> Plan, generate, and heal an executable E2E test suite from approved acceptance criteria (web and mobile).

Optional args: slug=<feature>, ticket=<id/url>, mode=interactive|autonomous|channel, channel=<id>, auto_continue=true|false, profile=business|hybrid|technical.

## Instructions

When the user asks to perform this workflow, execute the following steps:


# Test Loop Workflow

Goal: Turn approved ACs into an executable, traced E2E suite, and classify any failure as a repair or a real bug instead of leaving it to manual triage.

## Steps
1. Load scope:
   - `slug`, `operator_profile` (carried, not re-inferred), `AC-*`, SRS lanes, build/app target, matched testing skills for the target stack.
   - Run the driver preflight for each lane in scope (`quality-engineering-playwright-cli` / `quality-engineering-appium-mcp` `scripts/preflight.sh`); record lanes with no usable driver in `driver_blocked[]`.
2. Plan:
   - Run `specialist-test-planner` to produce `test_plan_path` and `selector_gaps`
     (the specialist's `PLAN:` becomes this workflow's `test_plan_path`;
     `SELECTOR_GAPS:` becomes `selector_gaps`).
   - BLOCKED (no stable `AC-*` trace) if no stable `AC-*` trace exists; route to `plan-feature`/`design-solution`.
   - BLOCKED (HALT: <trigger>) when the planner returns a `HALT:` trigger; ask before generating, never invent expected results.
3. Prepare selectors:
   - Run `specialist-testid-inserter` on `selector_gaps`; in interactive mode stop for approval on its `APPROVAL: required` file list; in autonomous mode pass `approved_production_edits` only when the operator granted it, else carry gaps forward as `selector_gaps_remaining[]`; every gap not in `INSERTED:` (unresolved `SKIPPED`, `BLOCKED`, or declined approval) also lands in `selector_gaps_remaining[]`.
   - For `lane: web`, build or extend one page object per screen per `quality-engineering-playwright-pom-generation`; record paths in `page_objects[]`; gaps a page object emits re-run the first bullet before step 4.
4. Generate:
   - One scenario per `specialist-integration-test-generator` call, seed-first, using the lane's driver (web: `playwright-cli`, fallback Playwright MCP; mobile: Appium MCP) per the driver skills' ladder; a `Test: BLOCKED` naming a missing page object routes back to step 3; a `Test: BLOCKED (driver)` lands in `driver_blocked[]`.
   - Skip scenarios whose only elements are in `selector_gaps_remaining[]`; list them under Selector Gaps Remaining, never generate against an unstable locator.
5. Run and heal:
   - Run the generated suite once; per failure, run `specialist-test-healer` with the run artifact; append its block to `heal_verdicts[]` as `{test, class, verdict, route, evidence}`.
   - `HEALED` only with `RERUNS: 3/3 green` and `ASSERTION_DELTA: none`; `REAL_BUG_DO_NOT_HEAL` appends to `real_bugs[]` and routes to `dev-fix`; `QUARANTINE_CANDIDATE` opens a ticket per `quality-engineering-flaky-triage` and appends `{test, ticket, expiry, bucket}` to `flake_quarantine[]`; `BLOCKED` with `ROUTE: testid-inserter` (no stable locator target) returns to step 3; `BLOCKED` (no evidence artifact) reruns the test once with tracing on and re-runs the healer; if still no artifact, it stays in `heal_verdicts[]` unresolved and is listed under `missing_evidence`.
   - Screenshot failures follow `quality-engineering-visual-baseline`: a baseline changes only through a reviewed diff with a named approver, never by `--update-snapshots` inside this loop.
6. Handoff:
   - Compute Automation Health per `quality-engineering-automation-health` and carry `release_confidence`.
   - Route to `verify-work` with the generated suite and any unresolved `real_bugs[]`.

## Runtime Contract
- Use after `implement-feature` reaches GREEN, or whenever ACs have E2E/mobile lanes without executable coverage.
- Required inputs: slug, stable `AC-*` trace, a runnable build/app target.
- Return BLOCKED (no build target or AC trace) only when the build target cannot be established or `AC-*` is missing.
## Handoff Payload
- `slug`, `operator_profile`, `test_plan_path`, `assumed_results[]`, `halt_triggers[]`, `page_objects[]`, `generated_tests[]`, `driver_blocked[]`, `heal_verdicts[]`, `flake_quarantine[]`, `selector_gaps_remaining[]`, `real_bugs[]`, `release_confidence`, outcome report, next workflow.
## Blocking Questions
- Ask max 3 at a time with a recommended default and 2-3 options.
## Output Template
```md
# Test Loop Report: [Name]
## Scope
## Plan
## Generated Tests
## Page Objects
## Selector Gaps Remaining
## Heal Verdicts
## Flake Quarantine
## Real Bugs Found
## Automation Health
feedback_loop_minutes: ; suite_reliability_pct: ; release_cadence: ; prod_escape_rate: ; release_confidence: high | medium | low
## Outcome Report
{schema_version: 1, run_id: "[run-id]", slug: "[slug]", workflow: test-loop, feature_status: implemented, started_at: "[timestamp]", completed_at: "[timestamp]", requirement_trace: {brd_objectives: [], requirements: [], acceptance_criteria: [], srs: []}, completed_evidence: [], missing_evidence: [], decision_needed: [], recommended_next_workflow: verify-work, cost: {source: unavailable}, agent: {identity: "[agent-identity]", model: "[model]"}}
## Next Workflow
verify-work | dev-fix
## Cost Report
Call `get_session_cost(workflow="test-loop")` before final handoff.
```