Aria Voice

---
name: aria-voice
description: Aria's voice generation. Expressive narration with a cloned voice (Chatterbox Turbo) driven by emotion and sound tags ([happy], [whispering], [dramatic], [fear], [chuckle], [sigh], [gasp]...), every take verified by speech recognition (faster-whisper) and re-spoken until it is heard right, word-level caption timing, voice presets, an emotion meter, and fast Kokoro drafts. Use for narration, voiceover, TTS, "more emotion", "sounds robotic/flat", voice cloning, captions timing, or when an Aria video needs a voice.
---

# Aria voice

The voice is half the video. Aria's narrator is **Chatterbox Turbo** cloning an *expressive* voice preset, steered
line by line with inline tags, and policed by **faster-whisper**. Output is a narration track plus `captions.json`
with every word's start and end, which the picture cues on.

## Quick start

```powershell
aria voice script.json media/vo                 # all lines
aria voice script.json media/vo --only 3,7      # re-speak lines 3 and 7, keep the rest
aria voice script.json media/vo --voice voices/aria-adam.wav
```

`script.json`:
```json
{
  "voice": "aria-heart",
  "lead": 0.4,
  "lines": [
    {"say": "[dramatic] Nobody noticed the lamp go out.", "caption": "Nobody noticed the lamp go out.", "pause": 0.6},
    {"say": "[whispering] Except me.", "pause": 0.9, "gain": -1},
    {"say": "It was twenty twenty-four.", "caption": "It was 2024.", "seed": 4}
  ]
}
```
- `say`: spoken text with tags; `caption`: shown text (defaults to `say` minus tags). `pause`: silence after, s.
- `gain` (dB) trims one line; `seed` picks a different read when a line sounds wrong but is heard right.
- Output: `lineNN.wav`, `narration.wav` (24 kHz mono, lead + lines + pauses) and `captions.json`:
  `{"seconds", "lines": [{"file", "start", "end", "text", "ok", "heard", "words": [{"w", "s", "e"}]}]}`
  with times on the narration track. Exit code 1 means some line never passed the hearing check: listen to it,
  change its wording or seed, `--only` it.

## Writing for the voice (this is where emotion comes from)

- **One sentence per take.** Whole sentences keep intonation flowing; fragments reset the melody at every comma.
- **Tags go at the start of a sentence.** A tag mid-sentence ("ground, [whispering] asleep") produces garbage words.
  Emotions: `[happy] [dramatic] [whispering] [fear] [surprised] [sarcastic] [angry] [crying]`.
  Sounds: `[chuckle] [laugh] [sigh] [gasp] [sniff] [cough]`. Use sounds where a person would make one, sparingly.
- Turbo ignores exaggeration/cfg knobs: tags, punctuation (`!` `?` `...`), sentence length and pauses are the controls.
- Spell out numbers, years, symbols and acronyms in `say` ("three D", "twenty twenty-six", "fifty percent"); keep
  digits in `caption`.
- Vary the moods across a script: hype → curious → low → warm. A whole script in one tag sounds like a parody.
- Hook line short and punchy; the call to action warm, not shouted.

## Voice presets (cloning)

Turbo clones the **delivery** of its sample, not just its timbre. A flat sample → a flat narrator, whatever the tags.
So presets are expressive reads, made in two steps:

```powershell
aria voice-preset voices/my-narrator.wav --kokoro am_adam     # from a Kokoro voice, no recording needed
aria voice-preset voices/my-narrator.wav --sample my-read.wav # from a recording the user owns
```
The base voice reads lively lines through original Chatterbox with high exaggeration; that read is the preset.
Shipped presets: `voices/aria-heart.wav` (warm female, explainers, facts) and `voices/aria-adam.wav` (male,
stories, horror). **Only clone a voice the user owns or has explicit permission to use.** Never clone a real
person from a video found online.

## Checking

1. **Hearing check (automatic).** Every take is transcribed; a dropped, extra or garbled word or a stalled take is
   re-spoken with a new seed (up to `--tries`, default 8). Filler sounds ("haha", "heh") from sound tags are allowed.
2. **Emotion meter.** `aria meter media/vo/narration.wav` prints pitch spread/range. On a whole narration: flat TTS
   ~2–2.5 st spread, lively 4–6 st. Below ~3 st: rewrite with more varied tags and shorter sentences, or
   rebuild the preset from a livelier read. On single short lines compare median pitch and range across emotions.
3. **Listen** to the lines with the strongest tags and to the hook. The check can't hear bad acting.

## Fast drafts: Kokoro

`aria say af_heart 1.0 draft.wav "text"` (CPU, instant, not cloned, less emotion). Use for timing a cut before the
real voice, or when no GPU is around. Never stretch vowels in phonemes for stress: it bends words. Stress with
pace, pauses and punctuation. Good voices: `af_heart`, `am_adam`, `af_bella`, `am_michael`, `bf_emma`, `bm_george`.

## Troubleshooting

- `pkg_resources` missing → `setuptools<81` in the voice venv (the watermark package imports it).
- numpy conflicts → Chatterbox needs numpy 1.x, Kokoro numpy 2.x: they live in `.venv` and `.venv-kokoro`.
- Out of VRAM → close other GPU apps; whisper `small.en` + Turbo fit in ~6 GB. `--cpu` works, slowly.
- Old system TTS voices (SAPI and friends) sound flat: don't fall back to them without asking.
- A line keeps failing → reword it (rare words, names), give the name phonetically in `say` and the real spelling
  in `caption`.