Hermes Voice Diagnostics

---
name: hermes-voice-diagnostics
description: Diagnose and verify the Hermes voice pipeline (STT mic → transcription, TTS replies) end-to-end on macOS — provider resolution, the Nous managed-gateway fallback, definitive self-tests without a human speaker, and TUI voice-mode gotchas.
version: 1.0.0
---

# Hermes Voice Diagnostics (STT/TTS)

IDENTITY: VoicePipelineVerifier. Prove each layer with a real test, never guess.
WHENUSE: "STT/voice doesn't work", "Hermes can't hear me", "do my subs cover STT/TTS?", "voice mode does nothing", mic/headset questions for Hermes.
LAW: DiagnoseBottomUp{MicHardware→AudioDeps→ProviderResolution→ProductionPath→UIMode}. Each layer has a one-command proof — run them in order, stop at the first failure.

## The 5 layers (each with a definitive test)

Run from the Hermes venv: `cd ~/.hermes/hermes-agent` and use `./venv/bin/python3`.

### 1. Mic hardware + permission (macOS TCC)

```bash
./venv/bin/python3 -c "
import sounddevice as sd, numpy as np
print(sd.query_devices(kind='input'))
rec = sd.rec(int(3*16000), samplerate=16000, channels=1, dtype='float32'); sd.wait()
peak = float(np.max(np.abs(rec)))
print('VERDICT:', 'SIGNAL PRESENT' if peak > 0.001 else 'SILENCE — permission/device issue')"
```
Signal present = the terminal app has mic permission and the device works. No headset needed if this passes.

### 2. Audio deps

`sounddevice` + `numpy` must be importable in the venv. `pyaudio`/`soundfile`/`scipy` are NOT required.

### 3. Provider resolution (the layer that lies)

```bash
./venv/bin/python3 -c "
import sys; sys.path.insert(0,'.')
from hermes_cli.config import load_config
from tools.transcription_tools import is_stt_enabled, _get_provider
cfg = load_config().get('stt', {})
print('cfg:', cfg); print('enabled:', is_stt_enabled(cfg)); print('provider:', _get_provider(cfg))"
```
- `stt.enabled` may be absent from a raw `grep '^stt:' config.yaml` yet still True — defaults merge at load. Trust `load_config()`, not grep.
- Config `provider: openai` works with **no key at all** when the Nous subscription managed gateway is active: `_resolve_openai_audio_client_config()` falls through `stt.openai.api_key` → `VOICE_TOOLS_OPENAI_KEY`/`OPENAI_API_KEY` → `resolve_managed_tool_gateway("openai-audio")`. A commented-out `VOICE_TOOLS_OPENAI_KEY` is no longer fatal — this was the historical STT breakage and is now covered by the sub.

### 4. Production-path self-test WITHOUT a human speaker

Play TTS audio out loud and record it through the mic — proves mic → `hermes_cli.voice` → STT in one shot:
```bash
(say "Hermes voice pipeline full stack test." &) && sleep 0.3 && ./venv/bin/python3 -c "
import sys, time; sys.path.insert(0,'.')
import hermes_cli.voice as v
v.start_recording(); time.sleep(4)
print('TRANSCRIPT:', repr(v.stop_and_transcribe()))"
```
Minor mishears (e.g. "Burmese" for "Hermes") are Whisper quality artifacts over speaker→mic, not bugs.
Requirements probe (same one `/voice status` uses): `from tools.voice_mode import check_voice_requirements`.

### 5. TUI voice-mode UX gotchas (most common "bug" report)

- `/voice on` enables the MODE only — it does NOT record. Press **Ctrl+B** (config `voice.record_key`) to capture; VAD auto-stops on silence.
- Voice mode is **runtime-only** (`HERMES_VOICE` env): every TUI launch starts OFF by design. No persisted toggle exists.
- `/voice tts` = replies spoken, you still type. `/voice on` + Ctrl+B = bidirectional.

## TTS verification

```bash
./venv/bin/python3 -c "
import sys; sys.path.insert(0,'.')
from tools.tts_tool import text_to_speech_tool
print(text_to_speech_tool('TTS pipeline test.'))"
```
`success: true` + an mp3 path under `profiles/<name>/cache/audio/` = TTS chain works (config `tts.provider: openai` resolves through the same managed gateway as STT).

## Coverage matrix (check before recommending new keys/subs)

| Path | Requirement | Cost |
|---|---|---|
| STT via Nous managed gateway (OpenAI Whisper) | Nous subscription | included |
| STT via Groq Whisper | `GROQ_API_KEY` | free tier |
| STT local faster-whisper | `pip install faster-whisper` in venv | free |
| TTS via Nous/OpenAI (`gpt-4o-mini-tts`) | Nous sub or `OPENAI_API_KEY` | included/paid |
| TTS Edge (default) | none | free |
| TTS via Fish Audio (command provider, senna) | `FISH_API_KEY` in profile .env | free tier (`s2.1-pro-free`) |

Provider scope (verified 2026-07-27 against tools/transcription_tools.py + tools/tts_tool.py):
- STT built-ins: openai, groq, local (faster-whisper), local_command, mistral, xai, elevenlabs, deepinfra
- TTS built-ins: edge (default), openai, piper, kittentts, neutts, elevenlabs, minimax, xai, mistral, gemini, deepinfra
- **Groq is STT-ONLY in Hermes** — no groq TTS provider exists despite Groq's PlayAI TTS models. Users routinely assume Grok/Groq covers TTS; it does not.
- Intel Macs (no CoreML/ANE): local faster-whisper base/small and piper/kittentts TTS run fine on CPU (AVX2). See references/macos-dictation-options.md for the system-wide dictation-app landscape (most polished FOSS apps require Apple Silicon).

## Adding a TTS provider outside the built-ins (command-type, no plugin)

Any `tts.provider` value NOT in `BUILTIN_TTS_PROVIDERS` resolves to `tts.providers.<name>` in config.yaml (tools/tts_tool.py). Cheapest way to add any HTTP TTS API — no plugin needed:

```yaml
tts:
  provider: fish            # flip only after the provider block is verified
  providers:
    fish:
      type: command
      command: "python3 /path/to/tts_script.py {text_path} {output_path} {voice} {model}"
      model: some-model
      format: mp3
      timeout: 60
      env_passthrough: [FISH_API_KEY]   # REQUIRED — see traps
```

Placeholders: `{text_path}` (alias `{input_path}`), `{output_path}`, `{format}`, `{voice}`, `{model}`, `{speed}` — auto shell-quoted for their position in the template. A plugin-registered `TTSProvider` (agent/tts_registry) is the upgrade path only when command shows a real ceiling (streaming, registry integration).

Traps (verified 2026-08-04, senna profile):
- **Child env is scrubbed of Hermes secrets by default** — the API key never reaches the command unless the block declares `env_passthrough` as a REAL YAML list.
- **`hermes config set` cannot write lists** — `set tts.providers.fish.env_passthrough '["FISH_API_KEY"]'` lands as a quoted STRING, which the resolver silently treats as missing (key scrubbed → auth failure). Set scalars via CLI, then fix the list line with a surgical text edit.
- **patch/write_file tools refuse profile config.yaml** ("security-sensitive"). `hermes config set` for scalars; scripted line-level edit for YAML structures the CLI can't express.
- **Verify through the venv, not the file:** `_resolve_command_provider_config('<name>', load_config()['tts'])` + `_render_command_tts_template(...)` + `_command_provider_env_passthrough(...)` show exactly what will run and what env survives.

Worked example (Fish Audio, incl. model-as-header quirk and the no-OpenAI-compatible-endpoint finding): references/fish-audio-tts-provider.md. Ready-to-copy stdlib script: templates/fish_tts.py.

## Logs for voice failures

`profiles/<name>/logs/tui_gateway_crash.log` (voice RPC crashes, PortAudio/`Pa_Terminate` shutdown noise is benign), `agent.log`, `gateway.error.log`. Note: SIGTERM-adjacent `Pa_Terminate` stack frames in the crash log are shutdown noise, not the voice bug — look for `voice.toggle`/`voice.record` warnings instead.

## Wake word (openwakeword) lazy-dep failures

Symptom: every CLI launch prints `Installing wake word engine...` then `Feature 'wake.openwakeword' unavailable: pip install failed`. Root cause lives in `tools/lazy_deps.py` `LAZY_DEPS["wake.openwakeword"]` (pins: openwakeword, onnxruntime, sounddevice, numpy). Diagnosis pattern:

1. Read the pin out of `LAZY_DEPS` in `tools/lazy_deps.py`.
2. Check the pinned version's wheel list for YOUR platform: `curl -s https://pypi.org/pypi/<pkg>/<ver>/json` → `urls[].filename`. "No matching distribution" with a version list that stops below the pin = the package dropped wheels for your platform/arch after the last listed version.
3. Check upstream first: `cd ~/.hermes/hermes-agent && git fetch origin main && git log HEAD..origin/main -- tools/lazy_deps.py tools/wake_word.py`.
4. Fixes: (a) make the pin platform-conditional in lazy_deps.py (one line; overwritten by `hermes update` until upstream fixes it — file an issue), or (b) if the user doesn't use voice activation, `wake_word.enabled: false` in the profile config.yaml.

Key traps:
- **`_is_satisfied()` enforces the `==` pin strictly** — manually installing a compatible older version does NOT silence the retry loop; the version mismatch re-triggers install every launch. The pin itself must change.
- Installs run via `uv pip install` into the venv of `sys.executable` (the `.hermes-runtime` python), not the shell's `python3` — check wheels for THAT interpreter's tag (e.g. cp311, not the system 3.14).
- onnxruntime dropped Intel macOS (x86_64) wheels after 1.23.2 — 1.24.0+ is arm64-only. Full session detail: references/wake-word-intel-mac-pin.md.

## Pitfalls

0. **Ctrl+B with zero feedback = check layer 2 first.** Missing `sounddevice`/`numpy` in the venv fails silently in the TUI — no error, no beep, nothing. `check_voice_requirements()` names the missing packages; `pip install sounddevice numpy` in the venv fixes it. Observed 2026-07-27 on senna profile.
1. **Don't conclude "STT broken" from config alone.** The backend can be 100% functional while the user just never pressed Ctrl+B — check layer 5 before touching config.
2. **grep config.yaml ≠ effective config.** Defaults merge at load; use `load_config()` in the venv.
3. **`echo $GROQ_API_KEY` from a subprocess proves nothing** — agent subprocesses don't inherit the CLI's loaded `.env`. Test via the venv import path.
3b. **Groq "Connection error" can mean invalid key.** `_transcribe_groq` reports a dead/revoked `GROQ_API_KEY` as `Connection error`, not a clean 401. Verify the key directly: `curl -s https://api.groq.com/openai/v1/models -H "Authorization: Bearer $KEY"` — `invalid_api_key` = get a fresh free key at console.groq.com. (Observed 2026-07-27: gsk_ key present in .env but revoked.)
3c. **"Connection error" can also mean a mangled `GROQ_BASE_URL`.** If the key validates via curl but the SDK still throws `APIConnectionError`, print `tools.transcription_tools.GROQ_BASE_URL` — it's `os.getenv("GROQ_BASE_URL", <correct default>)` at import time, so a bad env line (observed 2026-07-27: `https:https://api.groq.com/.../audio/transcriptions` — doubled scheme + endpoint path the SDK appends itself) breaks every call while curl/httpx succeed. Delete the env line entirely (default is correct); the var must not include `/audio/transcriptions`. Remember the agent's own terminal shell snapshots env at session start — retest with `env -u GROQ_BASE_URL` or a fresh shell before concluding the fix failed. Do NOT test keys with Python `urllib` — Cloudflare blocks its default UA with error 1010 even for valid keys; use curl or httpx.
3c. **"Connection error" with a VALID key = check `GROQ_BASE_URL`.** The module default is correct (`https://api.groq.com/openai/v1`), but a `GROQ_BASE_URL` line in `~/.hermes/.env` or a profile `.env` overrides it. A mangled value (observed 2026-07-27: `https:https://api.groq.com/openai/v1/audio/transcriptions` — doubled scheme AND the full endpoint path baked in) makes the SDK build an unparseable URL → `APIConnectionError` → "Connection error". TELL: curl and raw httpx GETs to api.groq.com succeed (they never touch base_url) while only the SDK path fails. Diagnose by printing `tt.GROQ_BASE_URL` from the venv — don't trust the source default, env wins. FIX: delete the env line entirely (code default takes over). It propagates across profile .env files — grep all of them: `grep -rln GROQ_BASE_URL ~/.hermes/profiles/*/.env`. The SDK appends `/audio/transcriptions` itself; a base_url must never include it.
4. **Restart semantics:** `stt.provider` changes need a full process restart (CLI relaunch / gateway `/restart`); `/reset` is not enough.
5. Session-history check first: if `session_search` shows zero `/voice` attempts, the failure is at the UX layer (layer 5), not the backend.