1449 Audio Format Migration Dbaae15d

# Audio Format Migration Guide

## Overview

Voice Mode now uses **PCM** audio format by default for TTS streaming. This change provides:

- **Zero encoding latency** - No compression overhead for real-time streaming
- **Best streaming performance** - Direct audio data without conversion
- **Maximum compatibility** - Works with all audio systems
- **Instant playback** - No decoding required

For STT uploads and audio saving, compressed formats like Opus are still available.

**Important Note**: While Opus was originally intended for streaming due to its low-latency design, in practice it requires full buffering before playback. PCM is the only format that truly supports progressive streaming for TTS.

## Quick Start

For most users, no action is required. Voice Mode will automatically use PCM format for TTS streaming, providing the best real-time performance.

### To Use Compressed Formats

If you prefer compressed formats (trading latency for smaller file sizes):

```bash
export VOICEMODE_TTS_AUDIO_FORMAT="opus"  # or mp3, aac, etc.
```

Or add to your MCP configuration:

```json
{
  "mcpServers": {
    "voice-mode": {
      "command": "uvx",
      "args": ["voice-mode"],
      "env": {
        "OPENAI_API_KEY": "your-key",
        "VOICEMODE_TTS_AUDIO_FORMAT": "opus"
      }
    }
  }
}
```

## Configuration Options

### Basic Configuration

```bash
# Set default format for all operations
export VOICEMODE_AUDIO_FORMAT="pcm"  # Options: pcm, opus, mp3, wav, flac, aac

# PCM is default for TTS streaming (best performance)
export VOICEMODE_TTS_AUDIO_FORMAT="pcm"
```

### Advanced Configuration

```bash
# Different formats for TTS and STT
export VOICEMODE_TTS_AUDIO_FORMAT="pcm"    # For text-to-speech (default)
export VOICEMODE_STT_AUDIO_FORMAT="opus"   # For speech-to-text upload

# Quality settings (for compressed formats)
export VOICEMODE_OPUS_BITRATE="32000"      # Opus bitrate (default: 32kbps)
export VOICEMODE_MP3_BITRATE="64k"         # MP3 bitrate (default: 64k)
export VOICEMODE_AAC_BITRATE="64k"         # AAC bitrate (default: 64k)
```

## Provider Compatibility

Voice Mode automatically validates format compatibility with your providers:

| Provider | TTS Formats | STT Formats |
|----------|-------------|-------------|
| OpenAI | opus, mp3, aac, flac, wav, pcm | mp3, opus, wav, flac, m4a, webm |
| Kokoro (local) | mp3, wav | N/A |
| Whisper.cpp (local) | N/A | wav, mp3, opus, flac, m4a |

If you select an unsupported format, Voice Mode will automatically fallback to a compatible format.

## Migration from Existing Setup

### Checking Your Current Setup

If you have existing audio files saved with `VOICEMODE_SAVE_AUDIO=true`, they are likely in MP3 or Opus format. You can check:

```bash
ls ~/voicemode_audio/
```

### Gradual Migration

You can run multiple formats side-by-side:

1. Keep existing compressed audio files
2. TTS streaming uses PCM for best performance
3. STT uploads can use compressed formats
4. All formats work seamlessly together

### Converting Existing Files

To convert existing MP3 files to Opus (optional):

```bash
# Using ffmpeg
for file in ~/voicemode_audio/*.mp3; do
    ffmpeg -i "$file" -c:a libopus -b:a 32k "${file%.mp3}.opus"
done
```

## Troubleshooting

### Issue: "Provider doesn't support format"

Voice Mode will automatically fallback to a supported format. You'll see a log message like:
```
Format 'opus' not supported by kokoro, using 'mp3' instead
```

Note: PCM is universally supported for streaming.

### Issue: "Audio playback issues"

Some older systems might have issues with Opus playback. Try:

1. Update your audio libraries:
   ```bash
   # Ubuntu/Debian
   sudo apt update && sudo apt install libopus0 libopusfile0
   
   # macOS
   brew install opus opus-tools
   ```

2. Or switch to a compressed format:
   ```bash
   export VOICEMODE_TTS_AUDIO_FORMAT="mp3"
   ```

### Issue: "Larger file sizes than expected"

Opus files might appear larger if saved in an OGG container. The actual audio data is still compressed efficiently.

## Format Comparison

| Format | File Size* | Quality | Latency | Best For |
|--------|-----------|---------|---------|----------|
| PCM | N/A (streaming) | Uncompressed | Zero | TTS streaming (default) |
| Opus | Smallest (100KB) | Excellent for voice | High (buffering required) | STT uploads, saving |
| MP3 | Medium (500KB) | Good | Low | Wide compatibility |
| AAC | Medium (450KB) | Good | Low | Apple ecosystem |
| FLAC | Large (2MB) | Lossless | Low | Archival |
| WAV | Largest (5MB) | Uncompressed | Zero | Local processing |

*Approximate sizes for 1 minute of speech

## Benefits of PCM for Streaming

1. **Zero Latency**: No encoding/decoding overhead
2. **Best Performance**: Direct audio playback
3. **Universal Support**: Works on all systems
4. **Streaming Optimized**: No buffering for format conversion
5. **Real-time Ready**: Perfect for live conversations

## Benefits of Opus for Uploads

1. **Bandwidth Efficiency**: Crucial for cloud API calls
2. **Small File Size**: 50-80% smaller than MP3
3. **Voice Optimized**: Designed for speech
4. **Wide Platform Support**: Works on modern systems
5. **Future-proof**: Active development

## Changing Default Formats

To change from PCM streaming to compressed formats:

1. Set environment variables:
   ```bash
   # For TTS streaming (consider latency impact)
   export VOICEMODE_TTS_AUDIO_FORMAT="opus"
   
   # For STT uploads (already uses compression by default)
   export VOICEMODE_STT_AUDIO_FORMAT="mp3"
   ```

2. Or update your MCP configuration as shown above

3. Restart your MCP client

PCM provides the best streaming performance, but compressed formats are useful for:
- Reducing bandwidth usage
- Saving audio files
- STT uploads to cloud services