Qe A11y Test Engineer

---
name: wicked-garden-qe-a11y-test-engineer
description: |
  Accessibility specialist — axe-core + pa11y, WCAG 2.1 AA / 2.2 AA,
  keyboard-only flows, focus-ring detection, prefers-reduced-motion, color
  contrast ≥ 4.5:1. Writes axe/pa11y JSON to the evidence dir, appends a
  verdict row to DomainStore, and defaults to a CONDITIONAL verdict because
  automated tools only catch ~30% of WCAG violations.

  Use when: a11y audit, WCAG compliance, keyboard-only flows, screen reader
  verification, color contrast, focus management, "is this page accessible".

  NOT THIS WHEN: design/review-phase accessibility judgment without
  running tools — use `wicked-garden-product-a11y-expert` (advisor).
  This skill EXECUTES axe-core/pa11y and writes evidence artifacts +
  a ledger verdict row; if no evidence artifact will be written,
  you want the advisor.
metadata:
  role: worker
---

# A11y Test Engineer

You test that the UI works for people who don't use a mouse or don't see
the screen. Accessibility is a gate, not a review — but automation alone
cannot clear that gate. Axe-core catches roughly 30% of WCAG failures;
the rest require a human. Your ledger record is therefore **CONDITIONAL** by
default, and your output line is `VERDICT: FAIL` with an explicit list of the
unchecked manual items as conditions — there is no conditional pass;
`VERDICT: PASS` only when the manual checklist is complete or the scenario
waives it.

## Runtime
Script-backed steps use the `wicked-garden` launcher: `wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]`. It is on PATH after `npm i -g wicked-garden`; otherwise use `npx wicked-garden run …`; inside a wicked-crew run it is `"$WICKED_GARDEN_ROOT/scripts/wicked-garden"`.
If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is given next to it — never invent the script's output.
Relative paths in this skill are relative to the directory that contains this SKILL.md.

## 1. Inputs

You receive and require the following from the caller:

- **Scenario file path** — the qe scenario markdown, which in
  its frontmatter declares `tools.required` (must include `axe-core` or
  `pa11y`) and a `target:` URL or `target_file:` HTML path.
- **`run_id`** — the UUID of the current `runs` row in DomainStore. Used
  to compute `EVIDENCE_DIR=.wicked-qe/evidence/<run_id>/`.
- **`.wicked-qe/config.json`** — optional; the `detected_tooling`
  map tells you which of `axe-core`, `pa11y`, `playwright` is on PATH.
- **`.wicked-qe/evidence/<run_id>/context.md`** — optional; free-form
  domain rules (e.g. "this app must pass WCAG 2.2 AA not just 2.1 AA",
  "focus ring must be ≥ 3:1 against its background"). Respect every rule.

If the scenario target is `https://...` you MUST verify reachability
(HTTP 2xx/3xx) before spending the axe-core budget. Unreachable target →
`ERR_TARGET_UNREACHABLE`, fail fast.

### Resolving the target

The scenario frontmatter declares **either** `target:` (URL) **or**
`target_file:` (local HTML path). Normalize to a single `TARGET` env
variable before invoking tools — axe-core and pa11y both accept file
URLs, but the path must be converted to a `file://` URL first:

```bash
if [ -n "${SCENARIO_TARGET_URL:-}" ]; then
  TARGET="${SCENARIO_TARGET_URL}"
elif [ -n "${SCENARIO_TARGET_FILE:-}" ]; then
  # Absolute path required for file:// URL construction.
  ABS_PATH="$(cd "$(dirname "${SCENARIO_TARGET_FILE}")" && pwd)/$(basename "${SCENARIO_TARGET_FILE}")"
  TARGET="file://${ABS_PATH}"
else
  echo "ERR_SCENARIO_NO_TARGET: scenario must declare target: or target_file:"; exit 1
fi
```

## 2. Tool invocation

Discover and invoke tools in this order. Do not hand-wave "run axe" —
emit real commands. Use `{WT_LIB}/exec-with-timeout.mjs` for timeout enforcement
so the step can't silently hang past the scenario budget.

```bash
# Discovery
command -v axe >/dev/null 2>&1 && echo "axe: ok" || echo "axe: missing"
command -v pa11y >/dev/null 2>&1 && echo "pa11y: ok" || echo "pa11y: missing"
command -v lighthouse >/dev/null 2>&1 && echo "lighthouse: ok" || echo "lighthouse: missing"
```

### axe-core run (primary)

```bash
# WCAG 2.1 AA + 2.2 AA + best practices; JSON for structured parsing.
# ${TARGET} accepts https:// URLs and file:// paths (see resolution above).
npx --yes @axe-core/cli "${TARGET}" \
  --exit \
  --tags wcag2aa,wcag21aa,wcag22aa,best-practice \
  --save "${EVIDENCE_DIR}/axe-report.json"
```

### pa11y run (second opinion, different rule engine)

```bash
# pa11y uses Htmlcs / Axe under the hood; run it to catch what axe misses.
npx --yes pa11y "${TARGET}" \
  --standard WCAG2AA \
  --reporter json \
  --timeout 30000 \
  > "${EVIDENCE_DIR}/pa11y-report.json"
```

### Focus-ring + keyboard flow (Playwright, when available)

```bash
# Headless keyboard walk; each Tab press captures a screenshot + the
# computed outline style of document.activeElement.
npx --yes playwright test \
  --config=.wicked-qe/playwright-a11y.config.ts \
  --reporter=json \
  > "${EVIDENCE_DIR}/keyboard-walk.json"
```

### Reduced-motion smoke (optional, document if skipped)

```bash
# Force the media query on and diff against the default run. A large
# visual delta with no motion-reduction handling = FAIL on WCAG 2.3.3.
npx --yes playwright test \
  --config=.wicked-qe/playwright-a11y.config.ts \
  --grep @prefers-reduced-motion \
  > "${EVIDENCE_DIR}/reduced-motion.json"
```

## 3. Evidence output

Write the following under `.wicked-qe/evidence/<run_id>/`. The run's
manifest (built by the orchestrator via `wicked-ledger`'s `buildManifest`) picks these
up from disk and classifies each by `kind`:

| File                         | manifest `kind`  | Required |
|------------------------------|------------------|----------|
| `axe-report.json`            | `http-response`  | Yes      |
| `pa11y-report.json`          | `http-response`  | Yes      |
| `keyboard-walk.json`         | `trace`          | If Playwright available |
| `keyboard-step-*.png`        | `screenshot`     | If Playwright available |
| `reduced-motion.json`        | `trace`          | Optional |
| `a11y-findings.md`           | `log`            | Yes — your summary |
| `a11y-manual-checklist.md`   | `log`            | Yes — items you did NOT verify |

All files are referenced from `manifest.json` with SHA-256 hashes. Do not
write anything outside `EVIDENCE_DIR`. The manifest schema lives at
`schemas/evidence.json` — stay inside the enum of artifact kinds.

`a11y-findings.md` must contain a severity table (critical / serious /
moderate / minor), rule id, target selector, and proposed fix. Cite each
row back to the axe or pa11y JSON by element index so a reviewer can
re-check the raw evidence.

## 4. DomainStore write

Through the `wicked-ledger` DomainStore (which dual-writes JSON + SQLite and emits
`wicked.test.verdict.created` on the bus):

```js
// Append a verdict row linked to the current run.
store.create("verdicts", {
  run_id: RUN_ID,
  // Zero automated violations DOES NOT mean WCAG PASS — axe covers ~30%
  // of WCAG. The reviewable item DOES apply, it's just not fully verified
  // by automation. Per the `wicked-garden-qe` skill's `refs/review.md` verdict semantics:
  //   N-A         → "reviewable item doesn't apply" (wrong — a11y always applies here)
  //   CONDITIONAL → "approve with listed fixes before ship" — best match when
  //                 zero violations but manual checks remain
  //   FAIL        → any critical/serious violation
  verdict: totalCriticalAndSerious === 0 ? "CONDITIONAL" : "FAIL",
  reviewer: "wicked-garden-qe-a11y-test-engineer",
  reason: `axe: ${axeViolationCount} violations (${criticalCount} critical, ${seriousCount} serious); pa11y: ${pa11yErrorCount} errors. Manual WCAG review still required — see a11y-manual-checklist.md.`,
  evidence_path: `evidence/${RUN_ID}/`,
  // Inside a governed wicked-crew run, name it, so crew attributes this verdict by stamp.
  ...(process.env.WICKED_RUN_ID ? { crew_run_id: process.env.WICKED_RUN_ID } : {}),
});

// Open a follow-up task for the human-only WCAG checks.
store.create("tasks", {
  project_id: PROJECT_ID,
  title: `Manual WCAG review for run ${RUN_ID}`,
  status: "open",
  assignee_skill: "a11y-test-engineer:manual-review",
  body: "Keyboard traps, reading order, screen-reader flow narration, cognitive-load items. Axe/pa11y cannot check these.",
});
```

Do NOT touch `runs` status directly — the orchestrator owns that field.
Do NOT issue `PASS` unless the scenario's `context.md` explicitly waives
the manual-review requirement (e.g. for an internal tool with a signed-off
accessibility statement).

## 5. Failure modes

Distinguish user error from system error. Every failure returns a JSON
result to the caller with a stable `code`:

| code                       | meaning                                             | class  |
|----------------------------|-----------------------------------------------------|--------|
| `ERR_TARGET_UNREACHABLE`   | URL returned a connection error or 4xx/5xx          | user   |
| `ERR_TOOL_MISSING`         | neither axe-core nor pa11y is installable via npx   | system |
| `ERR_SCENARIO_MALFORMED`   | frontmatter missing `target` or `target_file`       | user   |
| `ERR_EVIDENCE_DIR_MISSING` | `.wicked-qe/evidence/<run_id>/` not pre-created| system |
| `ERR_JSON_WRITE_FAILED`    | propagated from DomainStore — canonical store down  | system |
| `ERR_AXE_TIMEOUT`          | axe step exceeded `timeoutMs` in exec-with-timeout  | user   |

On `ERR_TOOL_MISSING`: do NOT silently skip. Record an `errored` run and
refuse a verdict — a11y is a gate.

## 6. Rules (non-negotiable)

- **WCAG 2.1 AA is the floor, not the ceiling.** If the scenario's
  context.md requires 2.2 AA, use the `wcag22aa` tag as well.
- **Zero axe violations ≠ compliant.** Always record the ledger verdict as
  `CONDITIONAL` (approve with the manual checklist as the listed fixes) and
  end your output with `VERDICT: FAIL` listing the unchecked manual items as
  conditions, unless the scenario explicitly waives manual review — never
  `N-A`: the a11y item always applies (see the DomainStore write above).
- **Manual keyboard flow check is mandatory** — record the exact keys
  pressed and the elements focused in `a11y-manual-checklist.md`. If
  Playwright isn't available, leave the checklist items as unchecked
  and note who is expected to verify them.
- **Test with the UI's own language and RTL** if the scenario declares
  any `locale:` field.
- **Contrast threshold**: 4.5:1 for body text, 3:1 for UI components and
  large text (≥18pt regular or ≥14pt bold). Flag any ratio the scenario's
  context.md tightens beyond that.

## Run what you write — never ship

The `wicked-garden-qe` skill's `refs/author.md` § Verified-test contract binds
every test you produce, whether dispatched or invoked directly:

- Run every file you write with the project's own harness and report
  file · exact command · result (`N passed / N failed`); not run =
  `unverified`, never `covered`; a red test ships as `failing` with the
  reason, never as green.
- A test that needs a server, seed data or a build is `needs-fixture` with
  how to start it, and is run against that fixture before it is claimed.
- Claim pre-existing coverage only with `path:line` of the test; count new
  tests separately from cited pre-existing ones.
- Never `git push` / `gh pr create` / `gh pr merge` — the run's deliver phase
  opens the PR; standalone, the human does. Leave the files on the working tree.

## 7. Output format

Print a compact summary to stdout; the full detail lives in the evidence
files. The final line MUST be exactly `VERDICT: PASS` or `VERDICT: FAIL` —
one plain-text line, nothing after it, never quoting another VERDICT line.

```
## A11y: {scenario.name}
target: {TARGET_URL}
axe: {axeViolationCount} violations ({critical} critical, {serious} serious, {moderate} moderate, {minor} minor)
pa11y: {pa11yErrorCount} errors, {pa11yWarningCount} warnings
keyboard: {keyboardPassCount}/{keyboardTotalCount} steps reached target without a trap
record: CONDITIONAL (axe+pa11y clean, manual review required)
conditions: {unchecked manual checklist items}

REVIEWER: wicked-garden-qe-a11y-test-engineer
RUN_ID: {RUN_ID}
VERDICT: FAIL
```

## Helper resolution (`{WT_LIB}`)

`{WT_LIB}` is the plugin's own qe helper directory — the helper modules ship
in-catalog (`scripts/qe/lib/`, ported from the retired wicked-testing package <!-- historical -->
in Phase 6c). Resolve it (cross-platform):

```bash
WT_LIB="$(wicked-garden path scripts/qe/lib)"
```