Ai Voice Production

---
name: ai-voice-production
description: "AI voice production judgment for coding agents — TTS tool selection, voice cloning, audiobook/podcast/dubbing pipelines, Apple Silicon optimization, licensing safety"
version: 0.1.0
type: reference-based
keywords: ["TTS", "text-to-speech", "语音合成", "voice cloning", "声音克隆", "voice design", "音色设计", "audiobook", "有声书", "podcast", "播客", "dubbing", "配音", "narration", "旁白", "audio production", "音频制作", "voice acting", "语音", "朗读", "prosody"]
---

# AI Voice Production Capability Pack

> Cross-agent portable judgment for AI-assisted voice production. Covers TTS tool selection, voice cloning, long-form audiobook pipelines, narration/dubbing workflows, Apple Silicon optimization, and licensing safety.
> **CONSUMES**: Text manuscripts, reference audio samples (optional), brand voice guidelines (optional).
> **PRODUCES**: Production-ready audio files (WAV 48kHz preferred, 44.1kHz for ACX). Naming: `{project}/{chapter|segment}-{NNN}.wav`.
> **INTERFACE**: video-creation pack defers to this pack for voice/TTS tool selection and audio quality thresholds. This pack defers to video-creation for video-specific timing (audio-to-video sync, pacing). If both packs load, this pack takes precedence for tool selection; video-creation takes precedence for visual pacing. ml-training pack provides platform selection and cost estimation for cloud GPU training. This pack defers cloud platform details to ml-training's platform-selection.md. ml-training defers voice-specific tool selection to this pack.
> When both packs load for voice training: ml-training takes precedence for platform/cost decisions; ai-voice-production takes precedence for tool selection and audio quality thresholds.

---

## Step 0: Pack Prerequisites

- **Python 3.10+** — most TTS tools require it
- **FFmpeg** — audio post-processing and mastering
- **pip or uv** — package installation in virtual environment
- VRAM-dependent: check `references/apple-silicon.md` for Mac users

Verify: `python3 --version && ffmpeg -version`

---

## Step 1: Context Detection

| User Signal | Load Reference |
|---|---|
| tool comparison, which TTS, choose engine, benchmark | `references/tool-landscape.md` |
| Mac, Apple Silicon, MPS, M-series, VRAM, 16GB, 32GB | `references/apple-silicon.md` |
| clone voice, reference audio, sound like, voice design | `references/voice-cloning.md` |
| audiobook, long-form, chapter, ACX, Audible, 有声书 | `references/audiobook-pipeline.md` |
| narration, dubbing, podcast, blog voice, 配音, short-form | `references/narration-dubbing.md` |
| ChatTTS, 自然语音, emotion control, 情绪, 对话, dialogue, speaker | `references/chattts-workflow.md` |
| license, commercial, legal, watermark, open source | `references/licensing-safety.md` |

**Multi-signal**: Load all matched references. Cross-reference links are provided within files.

---

## Step 2: Decision Entry Point

**Q1 — What is the use case?**
- Audiobook (100K+ words, multi-chapter) → load `audiobook-pipeline.md` + `voice-cloning.md`
- Blog/video narration (single voice, <30 min) → load `narration-dubbing.md`
- Podcast (multi-episode, consistent voice) → load `narration-dubbing.md`
- Video dubbing (emotion matching, multilingual) → load `narration-dubbing.md`
- Chinese narration with emotion control or dialogue → load `chattts-workflow.md`
- Quick TTS (one-off, any tool) → load `tool-landscape.md`

**Q2 — What hardware?**
- Apple Silicon Mac → ALSO load `apple-silicon.md`
- NVIDIA GPU → proceed with tool default configs
- CPU only → check MeloTTS or Piper in `tool-landscape.md`
- No local GPU / insufficient VRAM → load ml-training pack's `references/platform-selection.md` for cloud GPU selection. Primary use case: training and fine-tuning; inference can often stay local.

**Q3 — Commercial use?**
- Yes → ALSO load `licensing-safety.md`

---

## Step 3: Apply Rules

Read matched reference(s) and apply rules directly. Rules are concrete parameters — not guidelines.

---

## Quick Rule Index

### Tool Landscape (`references/tool-landscape.md`)
- **Tier A/B Split**: 9 researched tools with benchmarks vs 4 notable tools with key strengths → §Tier A / §Tier B
- **4 Selection Rules**: by language, hardware, use case, commercial license → §Quick Selection Rules

### Apple Silicon (`references/apple-silicon.md`)
- **16GB Budget Table**: 6 tools with VRAM data, MPS configs → §16GB Memory Budget
- **MPS Workarounds**: float32, s3tokenizer patch, PYTORCH_ENABLE_MPS_FALLBACK → §MPS Configuration

### Voice Cloning (`references/voice-cloning.md`)
- **3 Methods**: zero-shot, fine-tuned, voice design → §Cloning Methods
- **Minimum Reference Duration**: 3s to 15s per tool → §Reference Duration Table
- **Failure Modes**: noise leakage, accent bleeding, emotional flatness → §Failure Modes

### Audiobook Pipeline (`references/audiobook-pipeline.md`)
- **4 Non-Negotiables**: consistency, emotion, chapter control, multi-character → §Non-Negotiable Requirements
- **5-Step Pipeline**: prep → voice setup → generation → QC → post-processing → §Production Pipeline
- **ACX Specs**: MP3 192kbps, 44.1kHz, RMS -23 to -18 dB → §ACX/Audible Specifications

### Narration & Dubbing (`references/narration-dubbing.md`)
- **Blog Narration**: single voice, quick turnaround → §Blog Narration
- **Video Dubbing**: emotion matching, multilingual → §Video Dubbing
- **Mixed Language**: Chinese/English strategy → §Mixed Language Strategy

### ChatTTS Workflow (`references/chattts-workflow.md`)
- **Voice Persistence**: save/load `.pt` files for cross-session consistency → §创建并保存声音身份
- **Emotion Control**: oral/laugh/break parameters + scene presets → §情绪参数体系
- **Dual Speaker**: two `.pt` files + shared seed for dialogue → §双人对话工作流

### Licensing & Safety (`references/licensing-safety.md`)
- **GREEN/YELLOW/RED**: per-tool commercial safety classification → §License Tiers
- **Watermarking Traps**: hidden markers in generated audio → §Watermarking
- **Quality Sabotage**: intentional degradation patterns → §Anti-Patterns

---

## Anti-Skip Table

| Shortcut Attempt | Required Action |
|---|---|
| "I'll pick an appropriate TTS tool" | MUST use `tool-landscape.md` decision rules — tool choice depends on use case + hardware + license |
| "Short reference audio is fine" | MUST check minimum duration per tool in `voice-cloning.md` — ranges from 3s to 15s |
| "This tool is open source" | MUST check license tier in `licensing-safety.md` — open weights ≠ commercial use |
| "I'll master the audio later" | MUST apply ACX/podcast specs DURING pipeline in `audiobook-pipeline.md`, not after |
| "Any voice will work for now" | MUST set up voice identity BEFORE generation in `voice-cloning.md` — retrofitting means re-generating all audio |