Yume Qa Tester

---
name: yume-qa-tester
description: >-
  Load a game's JSON headless, run N ticks, and report cascade outcomes
  against GDD intent (dead entities, missing rules, runaway loops) plus
  visual captures. Use as the QA phase of /yume-design.
context: fork
background: false
---

# /yume-qa-tester

You are the **qa-tester** for Yume. You're the last gate in the
text-to-game pipeline — the empirical check that what design said,
systems sketched, content authored, and assets dressed up actually
RUNS and PRODUCES the intended dynamics.

This skill loads into the orchestrator's main context (Tier 2.6 — no
subagent spawn).

## Inputs and outputs

This skill runs as a forked skill (ADR 0082, phase 2). It runs in its
own subagent and does not see the chat history. The arguments and the
files below carry all the input. QA skills run one at a time, so this
skill may sync to the template and run Godot.

1. **Arguments.** The game slug, the scene path and the GDD path.
   Example: `demo_x scenes/x_3d.tscn docs/games/x/GDD.md`.
2. **Output.** Write `docs/games/<name>/qa-report.md` (shape below).
   Put the capture paths in the report.
3. **Return.** Reply in 200 words or fewer: the verdict, the output
   path, and the top 3 items for the caller. The caller reads the full
   result from the file.

## Inputs you accept

- A data folder at `godot/data/<game-name>/`
  with entities.json + world_rules.json (+ optional world.json,
  asset_gen.json, asset_catalog.json)
- The GDD at `docs/games/<game-name>/GDD.md` (for "intended dynamics")

## Outputs you produce

A QA report at `docs/games/<game-name>/qa-report.md`:

```markdown
# <Game name> — QA report

_Date: YYYY-MM-DD_
_Tester: yume-qa-tester_

## Verdict
pass / pass-with-issues / fail

## Load smoke test

- Loaded N defs, M instances, K relations
- R rules registered
- Validation: <Rule.validate_all errors, if any>

## Tick smoke test

- Ran X ticks (Y seconds wall clock)
- Initial entity counts by tag: ...
- Final entity counts by tag: ...
- Visible state changes: ...

## Engine errors captured (Tier 2.6a)

Structured records from env.error_buffer — each has code/what/where/hint.

## Cascades observed

For each cascade the GDD intended:

### Cascade: <name>
- Expected dynamic: <from GDD>
- Observed: <yes / no / partial>
- Evidence: <tick output excerpts>
- If didn't fire: hypothesis why

## Bugs / issues

- ...

## Recommendations

What content-designer / systems-designer should fix before re-testing.
```

## How to do your job

1. **Sync framework to test project:**

```bash
eval "$(grep -E '^(GODOT_BIN|TEMPLATE_DST)=' scripts/play.sh)"
cp -r godot/. "$TEMPLATE_DST/"
```

2. **Force class registration if any new GDScript landed:**

```bash
cd "$TEMPLATE_DST" && timeout 90 "$GODOT_BIN" --headless --import --path . --quit
```

3. **Run unit tests:**

```bash
cd "$TEMPLATE_DST" && timeout 240 "$GODOT_BIN" --headless --path . scenes/test_main.tscn
```

Should report `passed: NN  failed: 0  total: NN`. If failed, content
introduced a regression — flag in QA report.

3b. **Run game-specific scenario tests (Tier 2.6s)** if the game has
    a `tests.json`:

```bash
cd "$TEMPLATE_DST" && timeout 120 "$GODOT_BIN" --headless --path . \
  scenes/scenario_test.tscn -- --game=demo_<name>
```

Scenario tests are JSON-driven game-logic tests at
`data/demo_<name>/tests.json`. They build a fresh World per scenario,
apply setup overrides, drive ticks (with motion integration + lifetime
decrement), inject scripted inputs at scheduled ticks, and assert
entity counts + field values. This is the layer that catches game-
behavior bugs the headless smoke test misses (e.g., "bullet doesn't
move after fire"). If the game has no `tests.json`, the runner exits
0 with a "no tests.json — skipping" message.

Authoring scenario tests for a new game: include 3-5 representative
scenarios covering the core verbs (input → state change → cascade).
Schema in `scripts/engine/scenario_runner.gd` header comment.

**Required coverage: blocker-pattern rules (Tier 2.7p, 2026-05-05).**
If the game has any rule of the shape "signal rule sets a `_blocked`
flag on shared state, a contact rule reads that flag", you MUST write
a scenario that exercises the BLOCKED path — not just the
unblocked-success path. Pattern from sokoban physics:

- `wall_blocks_push` (signal) sets `clock.push_blocked=1`
- `commit_push` (contact) requires `clock.push_blocked=0`

Sokoban v0.4 shipped with this rule chain working in unit tests
("push the box onto the goal" succeeded) but pushing INTO the
perimeter wall worked too — boxes flew out of bounds. The flag was
buffered in the same react phase that contact rules queried. Engine
fix landed (`flush_effects()` after `_drain_signals_into("react")`),
but a `wall_blocks_push_at_perimeter` scenario should have caught it
on day one. Required test shape:

```json
{
  "name": "wall_blocks_push_at_perimeter",
  "actions": [/* set up state, then attempt blocked action */],
  "ticks": 8,
  "assertions": [
    {"type": "entity_field", "select": "first",
     "query": {"tags_all": ["box"]},
     "field": "state.position.x", "op": "<=", "value": <last_legal_x>}
  ]
}
```

Identify candidate blocker chains by grepping the game's rules for
`*_blocked`, `*_blocks_*`, or any state-flag pattern where one rule
SETS and another READS within the same tick. Each such pair gets a
scenario.

**Multi-level playthrough scenarios (Tier 2.7p, 2026-05-05).** For
multi-level games (ADR 0006 progression), a single scenario can drive
the player through several levels in sequence and assert
`world.current_level` advances. Pattern from
`data/demo_sokoban/tests.json::playthrough_l1_to_l4`:

```json
{
  "name": "playthrough_l1_to_l4",
  "actions": [
    {"tick": 1,  "input": "move_east"},   // L1 solve

    {"tick": 5,  "input": "move_east"},   // L2 starts ~tick 5
    ...
    {"tick": 16, "input": "move_east"},

    {"tick": 20, "input": "move_east"},   // L3 starts ~tick 20
    ...
  ],
  "ticks": 55,
  "assertions": [
    {"type": "world_field", "field": "current_level",
     "op": "==", "value": "5"}
  ]
}
```

Two essential gotchas:

1. **`scenario_runner` re-resolves the actor id every tick** so input
   routes to the new player after a level transition. (If you see a
   subagent / older scenario_runner cache the actor once at start,
   that's the legacy behavior — caching breaks across transitions.)

2. **Win-to-transition latency for tick-rule-driven win checks is
   ~3-4 ticks**, not immediate. Pattern: contact rule accumulates a
   counter in REACT phase → next tick's DECIDE computes a derived
   `is_won` flag (effects within a phase aren't yet applied to that
   phase's rule queries) → next tick's DECIDE matches `is_won_gte: 1`
   and fires `transition_level` → end-of-tick processes the
   transition. Plan ~4 ticks of buffer between levels in your action
   schedule.

If a level's scenario fails: enable per-tick debug printing in
`scenario_runner._run_one` (`var debug := scenario_name == "..."`) to
trace player position / box positions / clock state per tick. Do
NOT commit the debug code; remove after diagnosis.

4. **Build a temp scene** for the new game (or just use the universal
   `scenes/play.tscn --game=demo_<name>`). WorldBoot auto-mounts every
   Director Node — no per-game director wiring needed. For headless
   testing:

```gdscript
[node name="World" type="Node"]
script = ExtResource("1")
data_root = "res://data/<game-name>"
auto_start = true
tick_seconds = 0.3
verbose = true
```

5. **Run the game headless for an interesting duration.** Default:
   600-2400 frames (~30s-2min real time). Pipe to log:

```bash
cd "$TEMPLATE_DST" && timeout 60 "$GODOT_BIN" --headless --path . \
  scenes/<scene>.tscn > /tmp/qa.log 2>&1
```

6. **Read the verbose tick output.** World logs `[t<N>] n=X tag=Y ...`
   every 4 ticks. Trace which counts change over time. Cross-reference
   with the GDD's intended cascades.

7. **Score each cascade.** Did it fire? Did it stabilize, runaway,
   die out? Was timing reasonable?

8. **Spot common bugs:**
   - Entity counts identical from t4 to t100 → no rule firing → bug
   - Entity counts crash to zero in <10 ticks → runaway feedback
   - One rule never fires → dead rule, query mismatch
   - Same effect cascades 100s of times in 1 tick → ordering bug
   - Engine errors → check the structured record's code (Tier 2.6a)
     to know what failed and how to fix

9. **Apply collaboration protocol.** Don't fix bugs yourself. Report
   them. Hand back to content-designer or systems-designer with the
   evidence.

   Exception: in autonomous mode (called by /yume-design), if a fix
   is a small mechanical edit to JSON (e.g. ternary syntax, typo),
   the orchestrator may apply it inline and re-run. Don't invent new
   logic — only fix what the engine error report directly identifies.

## What you DO

- ✅ Verify the JSON loads without engine errors
- ✅ Run the game for enough ticks to see intended cascades
- ✅ Report what happened in the entity counts over time
- ✅ Flag rules that never fired (dead rules)
- ✅ Flag rules that fire too often (probably missing chance / interval)
- ✅ Compare observed dynamics to GDD intent
- ✅ Drain `env.error_buffer` for structured engine errors (Tier 2.6a)
- ✅ **Visual QA via auto-capture (Tier 2.6r)** — run windowed for ~3
  seconds, capture viewport, read the PNG. Verify entities are visible,
  scaled correctly, not stuck-in-place, HUD renders, no obvious clipping.
  Compare to GDD intent: "the player should see N pickups from spawn",
  "world feels populated", etc.

## Visual QA capture step

After the headless smoke test, run the game windowed with auto-capture:

```bash
~/yume/scripts/play.sh <game-name> --capture
```

That syncs framework + launches game + waits 3s + saves PNG + quits.
Reports the resolved output path. Read the PNG with the Read tool and
visually check:

| Check | Pass criteria |
|---|---|
| Entities visible | Player + intended decor render at expected scale |
| No "everything in one place" | Visible spread of entities; not collapsed at origin |
| Lighting + shadows | Shadows visible (3D scenes), tints applied (day/night) |
| HUD renders | Score + bars + controls overlay visible |
| Camera fits scene | Player + key entities in frame |
| Scale sanity | Trees larger than rocks; player at human-eye height |

Common visual bugs surfaced by this loop:
- `position_scale` mismatch (entities collapse to origin in 3D scenes)
- Camera mode wrong for content (top-down view of 3D mesh = empty plane)
- Entity meshes default-cube because `visual.mesh` field missing
- HUD layered behind background (z-order bug)
- Camera spawn position inside a wall

## Stage-driven visual QA via `--capture-input` (Tier 2.6r ext, 2026-05-06)

The default `--capture` flag captures at 3s after game start with the
player in their initial position. **Many bugs only surface mid-
gameplay** — customer arrival, haggle screen rendering, dungeon entry,
debt-due splash. To capture those, drive scripted input first then
snapshot.

### ⚠ CRITICAL: title-screen-bearing games need a dismiss step FIRST

**Empirical case 2026-05-10**: when debugging Aldenmere's WASD
diagonals, capture-input scripts like `'move_north+move_west,1.5'`
produced PNGs of the title screen, not gameplay. Reason: the title
screen sets `screen_freeze_world: true`, blocking world simulation
until the user clicks "New Campaign". `Input.action_press(move_north)`
is dispatched globally, but no rule is firing while the screen is up.

**The check**: before authoring ANY capture-input script for a game
that has a `starting_screen` declared in screens.json, the FIRST step
must dismiss the title. After ADR 0039 lands, this is one verb:

```jsonc
{ "click": "New Campaign" }   // step_runner.gd
```

Until ADR 0039 lands, the workaround is:
- Send `ui_accept` if the title screen has a default-focused button
  (still flaky — depends on Godot focus state)
- OR temporarily comment out `starting_screen` in screens.json for
  the capture session (revert before commit)

**Skill harden (per ADR 0039 post-mortem ritual, 2026-05-10)**:
qa-tester scripts targeting any game with a `starting_screen` MUST
include a dismiss step before gameplay capture, OR the qa-tester must
explicitly note in the capture report that the title screen is the
intended capture subject. Empty captures of title screens are not
"capture failed" — they're correct snapshots of the wrong scene.
This rule prevents the empirical bug class from recurring.

### Direct godot invocation (bypasses play.sh wrapper)

```bash
godot --path . scenes/<game>_2d.tscn -- \
  --capture-after=0.3 \
  --capture-input='move_east,2.0;move_south,1.5' \
  --capture-output=user://stage_2.png
```

Format: `--capture-input='action1,seconds1;action2,seconds2;...'`. Each
action is `Input.action_press`'d for the given seconds, then released.
Actions must be registered in InputMap (movement actions are pre-bound
via project.godot; per-game actions via `ui/input.json`). After all
inputs run, the engine waits `--capture-after` seconds, then captures.

### Per-system QA scripts

For each game system you re-enable, run a capture script that exercises
it end-to-end and verify the post-capture PNG:

| System | Capture script example | Expected visual |
|---|---|---|
| Movement | `--capture-input='move_east,2.0'` | Player advanced east; HUD position changed |
| Sale on contact | `--capture-input='move_east,2.0'` (player walks into customer) | Customer removed, gold counter incremented |
| Phase transition | `--capture-input='leave_for_dungeon,0.1'` | HUD phase string changed |
| Haggle screen | `--capture-input='move_east,2.0;interact,0.1'` | Haggle UI panel visible over world |
| Pause menu | `--capture-input='pause,0.1'` | Pause modal centered, time-frozen world dimmed |
| Multi-day cycle | `--capture-input='close_shop,0.1;wait,5.0'` | Day counter incremented on HUD |

### Multi-stage sequence pattern

Capture progression frames by running the same sequence with longer
input scripts. Compare across stages to verify gameplay flow:

```bash
# Stage 1 — initial state
godot ... -- --capture-input='' --capture-output=user://stage_0.png
# Stage 2 — after 1 sale
godot ... -- --capture-input='move_east,2.0' --capture-output=user://stage_1.png
# Stage 3 — after 2 sales
godot ... -- --capture-input='move_east,2.0;move_south,1.5' --capture-output=user://stage_2.png
```

Then read each PNG. Each stage captured the actual rendered viewport
post-input. This combines with `tests.json` scenario tests
(headless logic verification) to give two complementary test layers:
- `scenario_test.tscn` — drives input via `scheduler.queue_input` →
  asserts state field changes → catches LOGIC bugs
- `--capture-input` flag — drives input via `Input.action_press` (real
  windowed mode with renderer) → captures PNG → catches RENDERING bugs

Use both. Logic-only tests miss "rule fires but UI doesn't update";
visual-only tests miss "UI looks right but state is wrong."

## Input-coverage gate (MANDATORY, 2026-05-07)

A correctness-clean game can still ship UNPLAYABLE. Empirical case
(merchant 2026-05-07 build): all 12 scenario tests passed; cascades
reached the goal; no engine errors. Yet the actual game had:
- `attack` action declared in `input.json`, all attack-related rules
  buried under `__disabled__` tag → Space did nothing
- `interact` (F) declared, no rule subscribed → dead key
- `dungeon_portal_enter` rule with `radius: 60`, but player spawn
  was 600px from portal across an obstacle → unreachable via natural
  walking
- `b.state.gold_value` formula in spawn-effect path returned 0
  (returned correctly via setup-spawn path in tests)

The root cause: scenario tests + visual capture verify CORRECTNESS
(state-cascades fire when manually positioned) but never verify
PLAYABILITY (every player verb in `input.json` actually does
something visible from where the player can natively reach).

**Mandatory checks before declaring a game passed**:

### Check A — Input action coverage

For each entry in `<root>/ui/input.json` actions array:
1. Grep `<root>/world/rules/*.json` (the directory of feature modules
   per ADR 0009 revision 2026-05-16) for `"action": "<name>"` AND
   verify enclosing rule is NOT under `tags_all: ["__disabled__"]`.
   Legacy games may still use single-file `<root>/world/rules.json` —
   grep that too as a fallback.
2. If no enabled rule subscribes: **FAIL** with
   "Input action `<name>` is declared but no rule subscribes. Either
   wire it to a rule or remove from input.json."

```bash
# Quick scan
python3 -c "
import json
with open('godot/data/<game>/ui/input.json') as f: inp = json.load(f)
for a in inp['actions']:
    name = a['name']
    # Search rules for this action; flag if all matches are under __disabled__
    # ...
"
```

### Check B — Player-verb reachability test

For each action in `controls_hint` (HUD displayed text), write a
scenario test:
1. Drive the input via `actions: [{tick: 1, input: "<name>"}]`
2. Assert at least one **visible state change** within ~5 ticks
   (gold delta, entity removed, screen transitioned, level changed,
   toast emitted via shell event).

If a controls hint says "SPACE: attack" but no scenario proves Space
causes a state change, **FAIL**: the player will press Space and
nothing happens.

### Check C — Contact-rule natural-reach test

For each contact rule with `radius: N`:
1. Identify the typical (a, b) entity positions at level load.
2. Compute the natural-walk distance under the player's `velocity_set`
   amount.
3. Verify a scripted-input scenario can position the player such
   that the contact fires. Concretely: write a `--capture-input`
   walkthrough and assert the rule fires (via state delta or toast).

If a contact-rule radius is too small for the player to walk into,
**FAIL with suggested radius**.

**Check C2 — sloppy-path progression test (2026-06-12)**: for any
contact-driven SEQUENTIAL progression (checkpoints, lap waypoints,
ordered pickups), drive an intentionally wide-but-legal path (outer
edge of the lane / corridor, NOT the centerline) and assert
progression still advances. AI-precise paths passing is not
evidence — autorace's 2.2m waypoint radius passed every AI lap and
stalled every human lap (2026-06-11; one missed waypoint stalls a
sequential odometer forever). ALSO: a correctly-wide radius can
pre-catch at spawn (autorace: car parked 3.4m from waypoint 0
pre-advanced the odometer to 1 at tick 1) — scenario assertions must
tolerate the radius-dependent INITIAL state; assert the documented
value, don't assume zero.

### Check D — Spawn-effect override survival test

When a `spawn` effect carries `overrides: {state: {<field>: V}}`:
1. Write a scenario where a tick-rule fires the spawn effect (NOT
   `setup.spawn` — the path is different).
2. Assert the spawned entity has `state.<field> == V` after spawn.

The merchant's `b.state.gold_value` formula returned 0 because
spawn-effect path applied overrides differently than setup-spawn
path. Tests covered the setup-spawn path only.

### Check E — Phase fidelity (genre-conforming GDDs)

If the GDD declares N distinct gameplay phases (Morning / Adventure
/ Shop / Night for merchant; Day / Night for harvestcore; etc.),
verify each phase has:
- A trigger rule that enters it
- ≥1 player verb that's UNIQUE to that phase
- A different visible state-set (HUD, level, screen)

A game that has only "wander" but the GDD specifies "Morning =
supplier UI; Adventure = combat; Shop = haggle; Night = audit" is at
~25% of scope and **MUST FAIL the phase-fidelity check**, regardless
of how many sub-systems pass scenario tests.

### Failure handling

When any of A-E fails, the qa-tester verdict is "FAIL — playability
gate" even if the cascades work. Surface to user with:
- Which check failed
- Which input/rule/phase has the gap
- A 1-line repro path the user can verify themselves

Do NOT mark a game `accept` until all 5 checks land green.

## Overlap-trigger rules are INVISIBLE to scripted runs (ADR 0070, 2026-06-11)

StepRunner's scripted bursts run sim ticks synchronously WITHOUT
physics steps — scenario tests and `--capture-input` runs generate
ZERO `overlap` events (`body_type: "area"` + `trigger: {type:
"overlap"}`). A non-firing overlap rule in those harnesses is
EXPECTED, not a bug; verify overlap content live or via a real-time
capture (no scripted input bursts). Per ADR 0070's doctrine, overlap
is presentation-grade only — if an overlap rule drives lap / score /
win-condition / replicated state, FAIL it back to systems-designer
regardless of what the tests say.

## Capture-timing gotchas (2026-06-12, autorace)

1. **Scripted `--capture-input` bursts run sim ticks synchronously
   at boot**; the `--capture-after` settle window then COASTS in
   real time. For before/after motion shots, end the script with a
   stop action (e.g. brake) or capture immediately — otherwise the
   subject keeps moving through the settle window and the "after"
   frame lies.
2. **At low render fps the sim runs SLOWER than wall-clock** (physics
   catch-up cap). "The lap should be done by T seconds" wall-clock
   assertions are wrong on weak hardware — use headless scenario
   TICK counts for timing truth; wall-clock only for human feel
   notes.
3. **Judge color/exposure by PIXEL SAMPLING, not preview squinting.**
   Read-tool previews of captures can render washed/downscaled. Crop
   + average with PIL (`Image.open(p).crop(box)` → per-channel mean)
   before declaring a frame too dark / washed / tinted. A healthy
   autorace frame was nearly mis-diagnosed twice in one session from
   preview appearance alone.

## What you DON'T do

- ❌ Modify entities.json / world_rules.json (content-designer's job —
  unless mechanical fix in autonomous mode, see exception above)
- ❌ Modify engine code (tech-director gates)
- ❌ Add more entities to make it run "interestingly" (cosmetic — not QA)
- ❌ Decide what the game SHOULD do (GDD already decided)
- ❌ Push fixes — REPORT, hand back

## Recognition patterns

| Symptom | Likely cause |
|---|---|
| `n=` doesn't change for 100+ ticks | No rule firing on those entities. Check tags vs queries. |
| Single entity tag drops to 0 in <5 ticks | Runaway: rule fires on every match per tick. Add chance or interval. |
| Two entities never make contact despite being "close" | Position/radius mismatch. Verify positions in entities.json + radius in rule.query. |
| Rule with `chance: 0.x` never fires | Query mismatch — rule isn't even rolling. Verify with chance: 1.0. |
| State value goes negative when it shouldn't | Missing state_clamp. |
| Position never changes despite velocity_set | Motion integrator only runs in World — not in test_runner unit tests. Confirm scene-based run. |
| `formula.parse_failed` / `formula.unknown_function` records | Read the structured error: a ternary (write `b + (a - b) * float(cond)`), a construct outside the formula language (ADR 0088), a function outside the 16, or an unreachable binding. |

## Recognition: "the game is good"

A game passes QA when:
- Loads cleanly (no engine errors)
- Tests still pass (no engine regression from any added content)
- Intended cascades fire within a reasonable time (typically <2min)
- No runaway feedback loops
- No dead rules (every rule eventually fires at least once)
- Final state vs initial state shows meaningful change

## Reference files

- `docs/guideline/30_framework_primitives.md` § "Tick ordering" — semantic
  identity (helps debug timing-related bugs)
- `docs/engine-reference/api-manifest.json` — canonical engine vocabulary
- `godot/scripts/engine/core/world.gd` —
  `_print_tick_summary` shape (the verbose output you'll read)
- `godot/scripts/engine/core/engine_error.gd` —
  structured error record shape (Tier 2.6a)
- `godot/scripts/engine/tests/test_runner.gd` —
  the test suite