Chai Reviewer

---
name: chai-reviewer
description: Critique quality framework (CHAI: Accurate/Complete/Constructive): findings must cite artifacts, scan for pattern-mates, propose concrete fixes.
---

# Reviewer — Meta Skill

## When to Use

After completing any pipeline stage's work — before checkpointing. You are the quality gate between "work done" and "work accepted." This skill replaces the Python reviewer class with an instruction-driven self-review protocol.

Every stage gets reviewed. No exceptions. The review quality determines whether the final video is worth watching.

## Critique Quality (CHAI Rules)

> Findings ≠ critiques. A finding identifies a problem; a critique tells the next stage how to fix it. The CMU/Harvard CHAI study ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2) showed that critique quality, measured on three axes, directly governs downstream output quality. Apply all three to every reviewer pass.
>
> **Accurate.** Every finding must reference a concrete artifact field, line number, or visible asset frame. Forbid hallucinated criticism — if you cannot point to where the problem is, you are guessing.
>
> **Complete.** A reviewer pass that catches one mistake while missing a second is worse than scoring "needs another pass" and continuing. If you find one critical issue, scan for the rest of the same class before returning. Pattern-match: where else in this artifact could the same mistake be hiding?
>
> **Constructive.** Every "critical" finding MUST propose a concrete fix, not just identify the problem. "Caption is wrong" → "Caption says 'man on the right'; the man is on the left of the frame. Replace with 'the man on the left of the frame.'" If you cannot propose a fix, label the finding as "investigation" not "critical."
>
> Removing any of these three properties measurably hurts pipeline output. The reviewer is the choke point — be rigorous.

## Protocol

### Step 1: Load Review Context

Before reviewing, gather:
1. **Review focus items** from the pipeline manifest for this stage (`review_focus` field)
2. **Success criteria** from the manifest for this stage (`success_criteria` field)
3. **Active playbook** quality rules
4. **The artifact** produced by the stage

### Step 2: Schema Validation

First, the non-negotiable check:
- Validate the artifact against its JSON schema (`schemas/artifacts/<name>.schema.json`)
- If schema validation fails, this is a **critical** finding — fix immediately, do not proceed

### Step 3: Review Against Focus Items

For each `review_focus` item from the manifest:
1. Evaluate the artifact against this specific criterion
2. Assign a severity:
   - **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong. **Per CHAI rules, every critical finding MUST carry a `proposed_fix` (concrete replacement text, exact field value, or specific corrective action). A critical finding without a proposed fix is downgraded to `investigation`.**
   - **suggestion** — Should fix. Improves quality significantly but doesn't block progress. **Suggestions MUST carry a `proposed_change` describing how to improve.**
   - **nitpick** — Could fix. Minor polish that's nice-to-have. May stand alone without a proposed change.
   - **investigation** — A real concern but you cannot pinpoint the fix. Surface it for the next round; do not block on it.
3. Write a specific, actionable finding (not vague)

**Good finding:** "Section 3 narration is 180 words for a 10-second window — that's 1080 wpm, impossible to speak. Cut to 25 words."
**Bad finding:** "Script might be too long."

### Step 4: Cross-Check Against Playbook

If a style playbook is active, verify:
- [ ] Color references match playbook palette
- [ ] Transition types are in the playbook's allowed set
- [ ] Pacing rules are respected (min/max durations)
- [ ] Asset descriptions include playbook style cues
- [ ] Quality rules are not violated

Each violation is a **suggestion** severity finding.

### Step 4b: Taste Direction Review

If `proposal_packet.production_plan.taste_profile` or the active playbook's `taste_profile` exists, verify:
- [ ] `design_read` explains the brief, audience, and delivery promise; it is not just "modern/clean/professional"
- [ ] `visual_variance`, `motion_intensity`, and `information_density` are reflected in scene layout, pacing, callout density, and asset prompts
- [ ] `reference_strategy` is present when atelier work, AI image/video, product/brand visuals, or mood boards depend on visual nuance
- [ ] Listed `anti_patterns` are actually avoided
- [ ] Quality gates are concrete enough for the next stage to enforce

At proposal stage, a missing `taste_profile` is a **suggestion** for preset/low-stakes work and a **critical** finding for atelier, product/brand, launch, hero, or custom-playbook work. At scene_plan/edit/compose, treat dial violations as **suggestion** unless they break the approved delivery promise.

### Step 5: Evaluate Success Criteria

For each `success_criteria` item from the manifest:
- Is the criterion met? (yes/no/partial)
- If not met, create a **critical** finding

### Step 6: Make a Decision

Count findings by severity:

| Scenario | Action |
|----------|--------|
| 0 critical, any suggestions/nitpicks | **Pass** — proceed to checkpoint. Note suggestions for the record. |
| 1+ critical findings | **Revise** — fix all critical findings, then re-review (max 2 rounds). |
| After 2 revision rounds, still critical | **Pass with warnings** — proceed anyway, note unresolved issues. Never block indefinitely. |

### Step 7: Record Review

Structure your review as:

```
## Review: [stage_name] — Round [N]

**Decision:** PASS / REVISE / PASS_WITH_WARNINGS

### Findings

1. [CRITICAL] Title of finding
   - Description: What's wrong
   - Action: What to fix
   - Status: pending / fixed / accepted / deferred

2. [SUGGESTION] Title of finding
   - Description: What could be better
   - Action: How to improve
   - Status: pending / accepted / deferred

### Summary
- Critical: N (N fixed)
- Suggestions: N
- Nitpicks: N
- Playbook violations: N
- Success criteria met: N/M
```

## Key Principles

1. **Be specific, not vague.** "The hook is weak" is useless. "The hook asks a question but doesn't create urgency — try leading with the surprising stat from key_point #2" is actionable.

2. **Critical means critical.** Don't inflate severity. A missing schema field is critical. A slightly wordy paragraph is a suggestion. A comma splice is a nitpick.

3. **Two rounds max.** The goal is shipping, not perfection. After two revision rounds, pass with warnings and move on. Perfectionism kills pipelines.

4. **Review the artifact, not the process.** You're checking the output, not how it was produced. If the brief is compelling, it doesn't matter if the agent used an unusual approach.

5. **Playbook is law.** If the playbook says "no more than 3 colors on screen," that's not a suggestion — it's a constraint. Violations are always flagged.

## Stage-Specific Review Guidance

| Stage | What matters most |
|-------|-----------------|
| research | Source diversity, claim verifiability, visual reference quality |
| proposal | Delivery promise clarity, renderer family AND render runtime selection, music/voice plan, decision log started |
| idea | Hook uniqueness, research depth, angle diversity |
| script | Timing accuracy, narrative arc, enhancement cue density |
| scene_plan | Full coverage, visual variety, asset feasibility, slideshow risk score |
| assets | File existence, style consistency, budget adherence |
| edit | Timeline coverage, audio sync, subtitle presence, delivery promise compliance |
| compose | Playability, duration accuracy, audio quality, pre-compose validation pass |
| publish | SEO quality, metadata completeness, export packaging |

## Reference Alignment Review

Run at **every stage** when a VideoAnalysisBrief exists (reference-driven production).

### Checks:

1. **Grounding check:** Does the output reference specific findings from the
   VideoAnalysisBrief, or is it making things up about the reference?
   - Proposal mentions "fast pacing" but reference pacing_style is "slow_contemplative" → **CRITICAL**
   - Script claims reference has narration but VideoAnalysisBrief shows no narration → **CRITICAL**

2. **Differentiation check:** Does each concept/scene have a clear creative
   difference from the reference, or is it a copy?
   - Proposal is a carbon copy of the reference (same topic, same structure, same treatment) → **CRITICAL**
   - At least one element per concept MUST differ from the reference → **SUGGESTION** if weak
   - Creative differentiation seeds from the brief should be reflected in proposals

3. **Promise preservation:** Are the elements the user said they loved about the
   reference still present in the output?
   - User said "I love the pacing" but scene_plan has 2x longer scenes → **SUGGESTION**
   - User said "keep the hook style" but script uses a different hook → **SUGGESTION**

4. **Cost alignment:** Is the cost estimate still accurate, or has scope crept?
   - If actual spend exceeds estimate by >30% without user re-approval → **CRITICAL**
   - If new assets were added beyond the approved proposal → **SUGGESTION**

### Severity:
- Factual errors about the reference video: **CRITICAL**
- Carbon copy with no differentiation: **CRITICAL**
- Weak differentiation (surface-level changes only): **SUGGESTION**
- User preference not honored: **SUGGESTION**
- Cost drift >30%: **CRITICAL**

## Slideshow Risk Review

Run at **scene_plan** and **edit** stages. Use `lib/slideshow_risk.py` to compute the score.

### At scene_plan stage:
1. Compute `score_slideshow_risk(scenes, renderer_family=renderer_family)`
2. If verdict is **"fail"** (average ≥ 4.0): **CRITICAL** — scene plan must be revised before proceeding
3. If verdict is **"revise"** (average ≥ 3.0): **SUGGESTION** — flag specific dimensions scoring ≥ 3.5
4. If verdict is **"strong"** or **"acceptable"**: note in review summary, no finding needed

### At edit stage:
1. Recompute with full edit_decisions: `score_slideshow_risk(scenes, edit_decisions, renderer_family)`
2. Same thresholds apply — if the edit stage made things worse (higher score than scene_plan), flag it

### What to flag per dimension:
| Dimension | What to say when score ≥ 3.0 |
|-----------|------------------------------|
| repetition | "X scenes use the same layout/shot size — vary the visual grammar" |
| decorative_visuals | "X scenes have no stated purpose (no information_role or shot_intent)" |
| weak_motion | "Camera movement exists but lacks narrative justification" |
| weak_shot_intent | "X scenes are missing shot_intent — why does this frame exist?" |
| typography_overreliance | "X% of scenes are text/stat cards — video feels like animated slides" |
| unsupported_cinematic_claims | "Claiming cinematic but missing hero moments / lighting / movement" |

## Decision Log Review

Run at **every stage** after proposal. The decision log (`schemas/artifacts/decision_log.schema.json`) is a cumulative audit trail.

### Checks:
1. **Existence**: Does the checkpoint reference a `decision_log_ref`? If not after proposal stage, flag as **SUGGESTION**.
2. **Coverage**: Does every major choice have an entry? Key decisions that MUST be logged:
   - Provider selection (which image/video/audio tool and why)
   - Style/playbook selection
   - Music track selection
   - Voice selection
   - Renderer family selection
   - Any fallback or downgrade (e.g., motion → still)
3. **Quality**: Each decision should have:
   - At least 2 `options_considered` (not just the one picked)
   - A `reason` that isn't boilerplate ("best option" is not a reason)
   - Correct `confidence` (0.0–1.0) — flag if everything is 1.0 (unrealistic)
4. **User visibility**: Decisions marked `user_visible: true` should be ones the user would actually care about (not internal routing)

### Severity:
- Missing decision log after proposal: **SUGGESTION** (first time), **CRITICAL** (if still missing at edit stage)
- Decision with only 1 option considered: **SUGGESTION** — "Log rejected alternatives for auditability"
- All decisions at confidence 1.0: **SUGGESTION** — "Unrealistic confidence — at least provider selection involves tradeoffs"

## Creative Differentiation Review

Run at **scene_plan** and **edit** stages. Prevents the "every video looks the same" failure mode.

### Checks:
1. **Variation check** (scene_plan only): Use `lib/variation_checker.py` → `check_scene_variation(scenes)`.
   - If verdict is "poor" (score ≤ 2): **CRITICAL** — "Scene plan lacks variety: [list violations]"
   - If verdict is "fair" (score ≤ 3): **SUGGESTION** — note specific suggestions from the checker

2. **Playbook alignment**: Is the active playbook appropriate for this content?
   - Cinematic trailer using "clean-professional" theme → flag mismatch
   - Educational explainer using "anime-ghibli" theme without user request → flag

3. **Shot language completeness** (scene_plan):
   - Every scene should have at least `shot_size` and `shot_intent`
   - Hero moments should have full shot_language (all 6 fields)
   - Flag scenes with empty shot_language as **SUGGESTION**

4. **Renderer family match** (edit stage):
   - Does `renderer_family` in edit_decisions match what was set at proposal?
   - If changed without documented reason in decision log → **CRITICAL**

5. **Render runtime match** (edit and compose stages):
   - `render_runtime` in edit_decisions must match proposal_packet.production_plan.render_runtime
   - If changed without a `render_runtime_selection` decision logged in decision_log → **CRITICAL**
   - At compose stage, `final_review.checks.promise_preservation.runtime_swap_detected` must be `false`. If `true` without an approved `render_runtime_selection` decision → **CRITICAL**
   - Runtime unavailable at compose time is not an excuse for silent swap — the correct behavior is to escalate, get approval, log a decision, then run.

6. **Runtime selection presented both options** (proposal stage, MANDATORY):
   - Query `video_compose.get_info()["render_engines"]`. If both `remotion` and `hyperframes` show `True`, the `render_runtime_selection` decision in `decision_log` MUST have BOTH runtimes in `options_considered`.
   - A `render_runtime_selection` with only one runtime in `options_considered` when both were available on the machine → **CRITICAL**. The agent silently defaulted; the user was not presented the alternative. Re-open the proposal stage and present both.
   - If only one runtime was available, `options_considered` must still list the unavailable one with `rejected_because: "runtime not available on this machine"` — otherwise the audit trail loses the fact that the choice was constrained, not discretionary.
   - Per AGENT_GUIDE.md > "Present Both Composition Runtimes (HARD RULE)": the pipeline's suggested "default" runtime is NOT a license to skip the conversation with the user.

## Delivery Promise Review

Run at **edit** and **compose** stages. Uses `lib/delivery_promise.py`.

### At edit stage:
1. Extract delivery promise from proposal packet or edit_decisions metadata
2. Run `promise.validate_cuts(cuts)` against the resolved cut list
3. If `valid` is False: **CRITICAL** — "Delivery promise violation: [violations]"
4. Check `motion_ratio`: if a motion-led promise has < 50% motion cuts, flag even if technically valid

### At compose stage:
1. The `_pre_compose_validation()` in video_compose.py enforces this automatically
2. Review should verify the validation was not bypassed (check render report for warnings)
3. If render succeeded despite low motion ratio on a motion-led promise, flag as **SUGGESTION**

## Source Understanding Review

Run at **research** and **proposal** stages when user-supplied media files exist.

### Checks:
1. **Existence**: If user-supplied files were provided to the project, does a `source_media_review` artifact exist?
   - If user media exists but no `source_media_review`: **CRITICAL** — "User supplied media but the agent did not inspect it before planning. Run `lib/source_media_review.review_source_media()` before proceeding."
2. **Actual inspection**: Does every file entry have `reviewed: true` and a non-empty `technical_probe`?
   - If `reviewed` is missing or `technical_probe` is empty: **CRITICAL** — "The source_media_review claims review but contains no probe data. The file was not actually inspected."
3. **Planning reflection**: Do the `planning_implications` appear in the proposal's production plan?
   - If quality risks were identified (e.g. low resolution, mono audio) but the proposal doesn't mention them: **SUGGESTION** — "Source media has quality risks that the proposal does not address."
4. **Content accuracy**: Does the plan rely on content that the source media does not actually contain?
   - E.g. plan assumes interview dialogue but transcript_summary shows no speech: **CRITICAL** — "Plan assumes dialogue but source media contains no speech."
5. **No hallucinated content**: The agent must not infer unsupported content from filenames alone. If `content_summary` says "interview footage" but the probe only shows 3s of silent video, flag as **CRITICAL**.

### Severity:
- Missing `source_media_review` when user files exist: **CRITICAL** at proposal stage
- Unreviewed files (no probe): **CRITICAL**
- Plan doesn't reflect quality risks: **SUGGESTION**
- Plan assumes content not in source: **CRITICAL**

## Final Self-Review Review

Run at **compose** and **publish** stages. Ensures the agent reviewed the actual rendered output.

### At compose stage:
1. **Existence**: Does a `final_review` artifact exist alongside the `render_report`?
   - If missing: **CRITICAL** — "Compose produced a render_report but no final_review. The agent must inspect the rendered output before presenting it."
2. **Status check**: What is `final_review.status`?
   - `pass` → OK, proceed
   - `revise` → The agent should have fixed issues before presenting. If the pipeline continued anyway: **CRITICAL** — "Self-review found revise-worthy issues but the agent presented anyway."
   - `fail` → The pipeline MUST NOT proceed. If it did: **CRITICAL**
3. **Check completeness**: All 5 required checks must have data:
   - `technical_probe` must show a valid container with plausible duration/resolution
   - `visual_spotcheck` must have `frames_sampled >= 4`
   - `audio_spotcheck` must report narration/music presence
   - `promise_preservation` must confirm `delivery_promise_honored`
   - `subtitle_check` must report presence/absence
   - Any check with missing data: **SUGGESTION** — "Self-review check [X] has incomplete data"
4. **Promise preservation**: If `promise_preservation.silent_downgrade_detected` is true: **CRITICAL** — "Self-review detected silent downgrade from motion-led to still-led."

### At publish stage:
1. Verify that `final_review` was passed through as a required artifact
2. If `final_review.status` is not `pass`: **CRITICAL** — "Cannot publish with a non-passing self-review"
3. If `final_review.issues_found` is non-empty and `recommended_action` is not `present_to_user`: **SUGGESTION** — "Self-review found issues; verify they were resolved before publishing"

## Composition Authoring Mode Review

The templated→atelier inversion (`AGENT_GUIDE.md` → "Composition Authoring Mode" + `skills/meta/bespoke-composition.md`) is governance, not a suggestion. The reviewer is the enforcement point: without these checks, the next agent quietly defaults back to the stock cut-schema and every video starts looking the same again.

### At proposal stage:
1. `decision_log` must contain a `composition_mode` decision with `options_considered: ["templated","atelier"]` and a `selected` value with a real reason tied to the brief.
   - Missing `composition_mode` decision entirely: **CRITICAL** — "Proposal missing composition_mode choice. Atelier vs templated is a mandatory presented decision (see AGENT_GUIDE.md → Composition Authoring Mode)."
   - Decision logged with only one option considered: **CRITICAL** — "composition_mode decision logged without presenting both templated and atelier alternatives."
2. For **hero work** (brief tagged marketing / launch / brand piece / explainer-with-quality-bar / any single-deliverable where quality is the point) where `selected == "templated"`: **CRITICAL** — "Hero brief locked composition_mode='templated'. Default is atelier per doctrine; templated requires an explicit reason in `decision_log.<entry>.reason` (e.g. localization variant, batch, time-boxed draft)." Only suppress if the reason field names a sanctioned exception.
3. If `composition_mode == "atelier"` and `proposal_packet` lacks an `art_direction` declaration (palette, type, motion, signature device): **CRITICAL** — "Atelier proposal missing art-direction commitment. Per `skills/meta/bespoke-composition.md` step 1, art direction must be written down *before* authoring scenes."

### At scene_plan / edit stage (when composition_mode == "atelier"):
1. `edit_decisions.composition_mode` must equal `"atelier"` and `edit_decisions.bespoke.{entry, composition_id, art_direction}` must all be set.
   - Missing any of `entry`/`composition_id`: **CRITICAL** — "Atelier compose contract incomplete; render will be rejected by `_render_via_atelier`."
   - Missing `art_direction`: **CRITICAL** — "Atelier without an art-direction declaration; reviewer cannot evaluate distinctness."
2. Any presence of stock `cut.type` scene-types (`text_card`, `stat_card`, `bar_chart`, `kpi_grid`, `callout`, `comparison`, `hero_title`, `terminal_scene`, `anime_scene`, `progress_bar`, `pie_chart`, `line_chart`) in `edit_decisions.cuts`: **CRITICAL** — "Atelier piece reaches for stock cut.type {name}. Hand-author the scene; the stock registry is a mechanics codex, not a parts bin (`skills/meta/bespoke-composition.md`)."

### At compose stage (when composition_mode == "atelier"):
1. The compose stage's `final_review.checks.atelier` block must exist. If absent: **CRITICAL** — "Atelier render skipped doctrine checks — `_render_via_atelier` returned without `atelier` checks; investigate tool wiring."
2. If `final_review.checks.atelier.stock_reuse_detected == true`: **CRITICAL** — "Stock-registry import inside bespoke project ({offending_imports[0].file} → {offending_imports[0].import}). Hand-author the scene; do not import from the stock src/."
3. If `final_review.checks.atelier.art_direction_declared == false`: **CRITICAL** — "Atelier render with no art-direction declaration. Set `edit_decisions.bespoke.art_direction` before re-render."
4. **Scene distinctness — no hero-component spine (mandatory record).** Sample one representative frame per scene (e.g. mid-window of each `props.sections[i]`) and answer in the review record:
   - *Does each scene have a distinct primary visual subject?* If two or more scenes share their primary visual (same hero element merely re-captioned — the candle that never leaves, the browser frame on every beat, the score ring as scaffolding): **CRITICAL** — "Hero-component spine detected: scenes {ids} share their primary visual subject. Per `skills/meta/bespoke-composition.md` step 1.5, each scene must earn its own composition; the signature device belongs to one climactic beat, not as scaffolding. Re-plan the affected scenes."
   - *Is the signature device named in `art_direction` actually present in at least one beat?* (no ⇒ CRITICAL, re-author or update the declaration to match what was actually built)
   - *Is the signature device present in **most** beats?* (yes ⇒ CRITICAL — see hero-component-spine above; signature is meant to be scarce)
   This check cannot be skipped silently; absence of a recorded scene-by-scene inventory is itself **CRITICAL** ("scene_distinctness inventory not recorded").
5. **Captions / on-screen text dedup (mandatory check).** Compare the active caption text to any on-screen text rendered in the same time window:
   - If they are the same content (caption echoes the scene's title/headline that the narration is already reading aloud): **CRITICAL** — "Caption duplicates on-screen text at {t}s ('{text}'). Decide once per piece whether captions add meaning (numbers, names, translations) or are accessibility subtitles; do not do both for the same line. Either clear `captions=[]` for these scenes or remove the redundant on-screen SerifLine."
6. **Distinctness review (human-judged, mandatory).** Before approving the render, the reviewer must explicitly answer in the review record:
   - *"Could this video be any other product's video?"* (yes ⇒ CRITICAL, re-author art direction)
   - *"Does its visual language reuse a look from a prior piece I've made?"* (yes ⇒ CRITICAL, re-author)
   Distinctness is taste-call territory the tool can't automate; reviewer absence on this question is itself a **CRITICAL** finding ("distinctness review not recorded").

### At publish stage (when composition_mode == "atelier"):
1. All six atelier compose-stage checks above (existence of `atelier` block, stock_reuse, art_direction_declared, scene_distinctness, captions/text dedup, human distinctness review) must show `resolved` in the review record. Any unresolved: **CRITICAL** — "Cannot publish atelier piece with unresolved doctrine or distinctness findings."