talking-head-video

---
name: talking-head-video
description: >-
  Turn a raw 口播 / talking-head selfie video plus its script into a finished
  explainer, vertical 9:16 or landscape 横屏 16:9 — synced burned-in subtitles,
  animated knowledge graphics in three selectable styles (darkcard 深色科技卡
  default, watercolor 纸上水彩, inkline 白纸简笔; both style and orientation are
  confirmed with the user first), and a circular picture-in-picture avatar that
  shrinks the speaker to the top-left while a graphic or B-roll fills the screen
  during explanations. THE pipeline whenever someone has a clip of a person
  talking to camera and wants it "剪成成片", subtitles auto-added from a
  逐字稿/脚本, the 画面 to "切成图/弹图表/做大数字特效" on key points while the
  "人脸/人头缩小、放到角落、变成圆形头像/画中画/小圈圈", or Remotion/ffmpeg
  动态讲解图 cut into a 口播/讲解/知识科普短视频 for 视频号/抖音/B站/Shorts. Also
  covers standalone 知识卡/概念卡/章节卡 in any of the three styles (水彩/简笔画/
  深色卡) without a source video. Partial asks count: "帮我剪口播",
  "自动加字幕配动态图", "源视频和脚本都有帮我合成成片". NOT for: writing a 口播脚本,
  数字人/AI 主播 reading a script, multi-cam cutting, PPT→video, video
  compression, audio→text transcription, or subtitling an unrelated movie clip.
---

# Talking-Head Explainer Video

Build a finished 口播 video (vertical 9:16 by default, landscape 16:9 fully
supported — see 画幅 section) where the speaker is full-frame by default,
and during each explanation beat the speaker shrinks into a **circular avatar in
the top-left** while an **animated knowledge graphic** (chart / data cards /
comparison / flow / big-number) fills the screen. Subtitles are burned in and
corrected against the user's verbatim script.

This skill captures a real working pipeline. Read this whole file first, then the
references as needed. The design conventions exist for concrete reasons — keep
them unless the user asks otherwise.

## The core idea (why it's built this way)

The **original talking-head video is the spine** — its audio plays untouched the
whole time, so sync is never an issue. You only *overlay* graphics + a shrunk
avatar during "knowledge windows"; everywhere else it's just the full face. The
graphics render as **ProRes 4444 (alpha)** with a short fade at their edges, so
they **crossfade onto the face** instead of cutting to black.

```
full face ──┐         ┌── graphic full-frame + avatar top-left ──┐         ┌── full face
            │  cross  │   (one "knowledge window")               │  cross  │
  subtitles burned over EVERYTHING, the whole time
```

## Prerequisites (check / install once)

- **ffmpeg** with libass + prores_ks (Homebrew ffmpeg has these)
- **faster-whisper**: `python3 -m pip install faster-whisper`
- **Node + Remotion**: the template in `assets/remotion/` — `npm install` inside it.
  Remotion renders via a headless Chrome it manages.
- A CJK font libass can find (macOS: **PingFang SC** works out of the box).

## Workflow

Set up a working dir (e.g. `~/Downloads/<project>_build/`). Copy `assets/remotion/`
into it as `remotion/`. Then:

### 1. Inspect the source
`ffprobe` the clip for resolution/duration/fps. **Vertical 1080×1920 @ 30fps**
and **landscape 1920×1080 @ 30fps** are both first-class (see 画幅 section for
the parameter set of each). **开工前先和用户确认目标画幅**(通常跟源视频走,
但要问一句,别默默定;用户已明说的不再问). Extract a
frame (`ffmpeg -ss 2 -i src.mp4 -frames:v 1 f.jpg`) and look at where the face
sits — you need this for the avatar crop.

## 开工前先问用户(硬规则)

**制作视频前,必须和用户确认两个选项,给推荐但由用户拍板;用户已明说的不重复问:**
1. **画幅**:横屏 16:9 还是竖屏 9:16(参数表见下节);
2. **图卡风格**:darkcard / watercolor / inkline(见「图卡风格」节)。

## 图卡风格 · Graphics styles(三选一)

| 风格 | 一句话 | 适合 | 字体(fonts.ts 现成栈) |
|---|---|---|---|
| **darkcard** 深色科技卡(默认) | 深底、辉光、边框卡片、大数字 | 价格/参数/对比/新闻/评测 | 得意黑(DARK_TITLE)+ DIN(DARK_NUM)+ PingFang |
| **watercolor** 纸上水彩 | 米黄纸、水彩物件、铅字标题、物理点睛 | 生活类比/故事感/章节卡 | 汇文明朝体(WC_CN)+ Special Elite(WC_EN) |
| **inkline** 白纸简笔 | 纯白、细线"画出来"、大字+灰注、诗意收尾 | 原理/定义/一句话洞察 | 霞鹜文楷双字重(INK) |

工艺规范(色板/滤镜/动效语法/选择速查)在 `references/styles.md`,**写组件前先读**。darkcard
的六个 archetype 在 `references/design-system.md` + `assets/remotion/src/`;watercolor/inkline
的基础件在 `src/watercolor.tsx` / `src/inkline.tsx`;三风格各有完整示例在 `src/examples/`
(同一概念「边际成本」三种表达,当模板抄结构)。字体接线:`bash scripts/setup_fonts.sh <remotion项目>`
(自带 6 个免费字体;Anthropic 官方字体从本机 Claude.app 自动检测,**禁止随仓库/产物分发**)。
通用红线:**文字永不加做旧/毛边滤镜**,滤镜只作用于图形线条。

## 画幅 · Orientation parameter sets (both proven on real videos)

| 参数 | 竖屏 9:16 (1080×1920) | 横屏 16:9 (1920×1080) |
|---|---|---|
| Remotion Composition | width 1080, height 1920 | width 1920, height 1080 |
| 头像直径 / overlay 位置 | 300px / `40:120` | 380px / `44:44` |
| make_avatar.sh 参数 | `CROP≈1040:1040:0:250 SIZE=300` | `CROP≈780:780:580:20 SIZE=380`(居中脸源) |
| 字幕 ASS PlayRes / MarginV | 1080×1920 / 210 | 1920×1080 / 90(`generate_subs.py` 传 `--land`) |
| 图形安全区 | 避开左上头像 + y≳1540 字幕带 | 避开左上头像(x<470, y<470)+ 底部字幕带 |
| 引用他人视频的 @标签 | 右上角 | 右上角(躲开左上头像) |

横屏的典型场景:B站/YouTube 知识科普、新闻解析、"口播 + 整屏 B-roll/录屏"结构
(讲解时 B-roll 全屏盖脸、口播音频连续、头像浮左上)。竖屏参数是原始默认,
横屏参数来自金山岭项目实测。

### 2. Transcribe → timing
```
ffmpeg -i source.mp4 -ar 16000 -ac 1 -c:a pcm_s16le audio16k.wav
python3 scripts/transcribe.py audio16k.wav asr.json
```
`asr.json` gives word/segment timings. Trust the **timing**, not the text.

### 3. Build the segment map (the one real judgement step)
Read the user's script + `asr.json` and decide which time windows are
**knowledge windows** (get a graphic + avatar) vs **full-face** (rhetorical
beats, the hook, the conclusion — let the face carry emotion). A good rhythm
alternates: don't make every second a graphic. Each knowledge window maps to one
Remotion composition. Capture this as `segments.json` (see
`scripts/segments.example.json`).

Pick a graphic **archetype** per window (see `references/design-system.md` →
"Graphic archetypes"). Map the window's narration to the archetype's slots.

### 4. Build / adapt the graphics (Remotion)
In `remotion/src/`, each composition is one knowledge graphic. **Copy the closest
archetype component** (RecruitChart / MedicalCards / WetDry / PromptToDrug /
Edison / ThesisKeys) and rewrite its content. All of them import the shared
`theme.tsx` — use its tokens, `Backdrop`, `Kicker`, `Title`, `useEnter`,
`fadeUp`, `useClipFade`, and **respect `AVATAR_SAFE`** (keep the top-left clear).
Register each in `Root.tsx` with `durationInFrames = round(windowSeconds * 30)`.

Iterate fast with stills before full renders:
`npx remotion still src/index.ts <Comp> out.png --frame=<n>`

**Critical layout rule:** keep all graphic content **above y≈1540**. The bottom
~360px is reserved for subtitles. Put payoff callouts at `bottom: 380–430`, not
`bottom: 150`. (This was the #1 bug on the first build.)

### 5. Render graphics to alpha clips
```
bash scripts/render_graphics.sh remotion clips <Comp1> <Comp2> ...
```
Renders ProRes 4444 `.mov` (alpha) into `clips/`. Alpha is required for the edge
crossfade.

### 6. Make the circular avatar
```
bash scripts/make_avatar.sh source.mp4 avatar.mov 1040:1040:0:250 0x22D3A8
```
Tune the crop so the **whole head incl. chin** sits in the circle — wider crop =
smaller face. Validate on one frame first (the script prints where it wrote).
Common mistake: too tight a crop clips the chin.

### 7. Subtitles (corrected)
```
python3 scripts/generate_subs.py asr.json subs.ass
```
**Edit `CORRECTIONS` in the script per video** — whisper mis-hears brand names /
English terms / 同音字 (e.g. 施实验→湿实验, 礼来, Indeed, Verve, 胰腺癌, PPT).
Skim `asr.json` and add the wrong→right pairs before running.

### 8. Composite → final
Write `segments.json` (source, avatar, subtitles, output, and the windows), then:
```
python3 scripts/build_composite.py segments.json
```
It generates the ffmpeg filtergraph and renders `final.mp4` (graphics overlaid in
their windows + avatar top-left during windows + subtitles burned + original
audio). Use `--dry-run` to inspect the filter first.

### 9. QA (always)
Extract a frame from the **final mp4** at the middle of every segment type and
look at each:
```
ffmpeg -ss <t> -i final.mp4 -frames:v 1 qa_<t>.jpg
```
Check, per frame: avatar shows the whole face and sits in the clear top-left zone;
subtitle doesn't collide with graphic content; graphic content isn't cut off;
no black flash at window boundaries (mid-crossfade dark is fine). Fix, re-render
only the affected clip(s), re-composite.

## Editorial conventions worth keeping

- **Don't make it all graphics.** The face carries the hook and the emotional
  conclusion. Graphics are for facts/data/structure.
- **One subtitle correction pass is mandatory** — uncorrected ASR (wrong brand
  names, 同音字) instantly reads as sloppy.
- **Watch the narrative framing in graphics.** If the speaker's claim is a slight
  oversimplification you can't change (it's recorded audio), make the *graphic*
  text accurate so it doesn't contradict knowledgeable viewers. Likewise, if a
  segment risks an unintended read (e.g. "AI replaces humans"), add a bridging
  line in the graphic that steers toward the speaker's actual point.

## Reusable assets in this skill

- `assets/remotion/` — full Remotion project: `theme.tsx` (design system) + 6
  archetype components + `Root.tsx`. Copy into the project and adapt.
- `scripts/transcribe.py` — faster-whisper word timings
- `scripts/generate_subs.py` — ASR → corrected styled `.ass`
- `scripts/make_avatar.sh` — circular alpha PiP avatar (parameterized crop/ring)
- `scripts/render_graphics.sh` — batch ProRes-4444 render
- `scripts/build_composite.py` — segments.json → ffmpeg master composite
- `scripts/segments.example.json` — the segment-map schema
- `references/design-system.md` — palette, tokens, avatar/subtitle geometry, and
  the graphic archetype catalog. Read before designing graphics.