Jaw Video

---
name: jaw-video
description: "Code-based video create, edit, render, preview, and validate. Defaults to Remotion for JSON/React timelines, routes HyperFrames HTML video, TTS/STT captions, FFmpeg transforms, existing-footage editing, and docs-first provider/video-understanding workflows."
---

# Code-Based Video Skill

Route video work before rendering. Remotion is the implemented default for
JSON/React timeline videos. HyperFrames is the HTML/CSS/GSAP route for
agent-authored compositions. FFmpeg handles deterministic media transforms.

Triggers: "video", "Remotion", "HyperFrames", "animation", "mp4", "render video",
"slides to video", "HTML video", "edit this video", "subtitle video",
"video summary", "image to video", "presenter video".

Core rule: choose the route, load the matching reference, produce route-specific
artifacts, and validate the output. A successful command is not enough.

Authoring discipline: keep the main skill short, move deep engine rules into
references, and avoid copying large third-party docs into the active prompt.

---

## Route First

Read `reference/route-selector.md` when intent is not a simple Remotion render.

| User intent | Route | Status | Read |
|---|---|---|---|
| React/JSON timeline, branded explainer, TTS, captions, slide-to-video | Remotion | Active default | `reference/remotion.md` |
| HTML/CSS/GSAP composition, kinetic typography, premium tech briefing | HyperFrames | Active smoke route | `reference/hyperframes.md` |
| Authored narration, script-first subtitles, TTS-ready caption sidecars | TTS captions | Active via script JSON + Remotion draft timeline | `reference/tts-captions.md` |
| Audio/video transcription, diarized captions, subtitle generation, word-timed captions | STT captions | Active via progrok HTTP STT + FFmpeg prep | `reference/stt-captions.md` |
| Trim, concat, transcode, proxy, audio, reframe, deterministic media ops | FFmpeg utility | Active utility | `reference/validation.md` |
| Cut/polish existing footage with transcript, EDL, subtitles, overlays | Existing-footage editing | Docs-first | `reference/existing-footage-editing.md` |
| New generative video clips, image-to-video, cinematic model output | Provider video generation | Deferred docs-first | `reference/route-selector.md` |
| Presenter/avatar/talking-head video | Presenter/avatar video | Deferred docs-first | `reference/route-selector.md` |
| Search, summarize, index, or clip moments from existing video | Video understanding | Deferred docs-first | `reference/route-selector.md` |
| Browser-native media capture or MediaRecorder-style render | Browser media | Watchlist | `reference/route-selector.md` |

Default choice:

- Use **Remotion** unless the user asks for HTML-first authoring, raw media
  editing, provider generation, presenter/avatar video, or video search.
- Use **HyperFrames** for agent-editable HTML/CSS/GSAP compositions and premium
  tech briefings.
- Use **FFmpeg** for deterministic transforms.
- Treat provider, presenter, video-understanding, JSON-template, and browser
  media routes as docs-first/watchlist until a verified harness exists.

Do not promise render support for docs-first or watchlist routes. State the
missing harness or provider requirement.

---

## Visual Quality Gate

For any non-trivial original video, read:

- `reference/visual-direction.md`
- `reference/style-packs.md`
- `reference/visual-quality.md`

Workflow:

1. Perform a design read: audience, platform, palette, typography, density, and
   motion intensity.
2. Pick a style pack or define one.
3. Author the composition/edit.
4. Render or produce a preview.
5. Validate with media evidence plus sampled frames or snapshots.
6. Report a short visual QA verdict.

Hard rule: style presets are direction; validation gates are law.

---

## Remotion Quick Start

Use Remotion for the implemented JSON timeline pipeline.

```bash
node skills_ref/jaw-video/scripts/pipeline.mjs \
  --timeline timeline.json \
  --output /tmp/remotion-render
```

Common commands:

| Task | Command |
|---|---|
| Render | `node scripts/pipeline.mjs --timeline <path> [--preset Landscape-1080p]` |
| Render with TTS | `node scripts/pipeline.mjs --timeline timeline.draft.json` |
| Skip TTS | `node scripts/pipeline.mjs --timeline timeline.draft.json --skip-tts` |
| TTS only | `node scripts/tts.mjs --batch timeline.draft.json [--provider progrok]` |
| TTS single | `node scripts/tts.mjs --text "안녕하세요" --output /tmp/tts.mp3 --provider progrok --language ko` |
| TTS voices | `curl http://127.0.0.1:18645/v1/tts/voices` |
| TTS captions | `node scripts/tts-captions.mjs --script script.json --output /tmp/video-tts-captions` |
| STT captions | `node scripts/stt-captions.mjs --input input.mp4 --output /tmp/video-stt --language ko --diarize --word-timestamps` |
| Preview | `cd remotion-project && pnpm exec remotion studio` |
| Validate | `node scripts/validate-artifact.mjs /tmp/remotion-render/TimelineVideo.mp4 --preset Landscape-1080p` |

Read `reference/remotion.md` for timeline authoring, presets, TTS, components,
animation rules, and validation details.

Remotion animation rules:

- Use `useCurrentFrame`, `interpolate`, `spring`, and `<Sequence>`.
- Do not use CSS transitions, CSS animations, or Tailwind animation classes.
- Keep generated render outputs outside `skills_ref/video`.

---

## HyperFrames Quick Start

Use HyperFrames when the deliverable should be authored as HTML/CSS/GSAP.

```bash
npx --yes hyperframes doctor
npx --yes hyperframes init /tmp/my-video --example blank --non-interactive --skip-skills --resolution landscape
npx --yes hyperframes lint /tmp/my-video
npx --yes hyperframes render /tmp/my-video --output /tmp/my-video/renders/out.mp4 --format mp4 --quality standard --fps 30
ffprobe -v error -show_format -show_streams -of json /tmp/my-video/renders/out.mp4
```

Before rendering, author the composition. Rendering an untouched blank scaffold
is not valid proof.

Read `reference/hyperframes.md` for production loop, premium tech briefing
style, GeekNews smoke recipe, and verification gates.

---

## FFmpeg Utility Route

FFmpeg is the active deterministic media utility route.

Examples:

```bash
# Extract segment
ffmpeg -i raw.mp4 -ss 00:12:30 -to 00:15:45 -c copy segment_01.mp4

# Concatenate listed files
ffmpeg -f concat -safe 0 -i concat.txt -c copy assembled.mp4

# Proxy for faster editing
ffmpeg -i raw.mp4 -vf "scale=960:-2" -c:v libx264 -preset ultrafast -crf 28 proxy.mp4

# Extract audio for transcription
ffmpeg -i raw.mp4 -vn -acodec pcm_s16le -ar 16000 audio.wav

# Normalize audio
ffmpeg -i segment.mp4 -af loudnorm=I=-16:TP=-1.5:LRA=11 -c:v copy normalized.mp4
```

For cut-heavy editing, subtitles, transcript-aware decisions, or output
timeline EDLs, read `reference/existing-footage-editing.md` instead of treating
the task as raw FFmpeg snippets.

---

## Validation

Read `reference/validation.md` before declaring a video task complete.

Minimum route evidence:

| Route | Evidence |
|---|---|
| Remotion | output path, non-zero MP4, `validate-artifact.mjs` or ffprobe, sampled/still frame for non-trivial visuals |
| HyperFrames | doctor/lint/render, ffprobe, sampled frames or inspect output, visual QA verdict |
| TTS captions | script JSON, normalized narration, `captions.remotion.json`, draft/final timeline, TTS manifest or skip-TTS rationale, non-silent audio proof when TTS is expected, alignment proof when TTS changes duration, rendered frame proof if overlaid |
| STT captions | raw transcript, normalized transcript, SRT/VTT, `captions.remotion.json`, timestamp checks, rendered frame proof if overlaid |
| FFmpeg | input/output ffprobe, expected duration/dimensions/codec, output inspection |
| Existing footage | EDL/timeline artifact, transcript cache when speech matters, subtitle-last proof, cut-boundary checks |

Command succeeded != done. Completion requires route-specific evidence.

---

## Workspace Policy

Read `reference/workspace-policy.md` for source/output boundaries.

- `skills_ref/video` is source, not workspace.
- Generated video/audio/caption outputs go to `/tmp` or a project output folder.
- Do not commit generated MP4/WebM/GIF/audio unless explicitly requested.
- Third-party reference repositories are idea/reference material, not active
  runtime code.

---

## Reference Map

| File | When to read |
|---|---|
| `reference/route-selector.md` | ambiguous intent, docs-first routes, provider/understanding/presenter requests |
| `reference/remotion.md` | Remotion timeline, TTS, components, animation, render pipeline |
| `reference/hyperframes.md` | HTML/CSS/GSAP compositions and premium tech briefings |
| `reference/tts-captions.md` | authored narration, script-first subtitles, TTS-ready caption sidecars |
| `reference/stt-captions.md` | audio transcription, diarization, subtitles, word-level timing, Remotion caption JSON |
| `reference/existing-footage-editing.md` | transcript/EDL/subtitle/cut-boundary editing |
| `reference/visual-direction.md` | design read before non-trivial original video |
| `reference/style-packs.md` | first-pass video aesthetics and banned cliches |
| `reference/visual-quality.md` | anti-slop frame, typography, layout, and motion rules |
| `reference/validation.md` | proof bundles and completion gates |
| `reference/workspace-policy.md` | generated media, reference repos, and commit policy |
| `reference/tts-integration.md` | provider auth, voiceControl, speed, caching |
| `reference/components.md` | Remotion component library details |

---

## Dependencies

- Node.js 20+, pnpm, ffmpeg, ffprobe
- Remotion dependencies under `~/.jaw-shared/remotion/node_modules/`
- Bootstrap Remotion with `node scripts/ensure-remotion.mjs`
- HyperFrames via `npx --yes hyperframes ...`
- TTS via local progrok proxy `/v1/tts` by default; run `progrok proxy`
  or use `PROGROK_BASE_URL` when the proxy is on another port.
- TTS caption scripts may use `captionText` to display English proper nouns
  while narration stays provider-friendly. Captions auto-wrap with newline
  breaks and default to a smaller lower-third size.
- TTS captions via script JSON, existing TTS batch, Remotion caption sidecars
- STT captions via local `progrok` proxy `/v1/stt`, plus ffmpeg/ffprobe
- TTS: `GEMINI_API_KEY` for Gemini, `SUPERTONE_API_KEY` for Supertone,
  none for Supertonic