Skip to content
Back to skills

Sound Ear

ASecurity

Lets Claude 'hear' audio files. Turns songs, vocals or video soundtracks into numbers and pictures with a bundled script, then finds problems with timestamps: clicks, clipping, abrupt cuts, pitch glitches, off-pitch notes, wobbly notes, harsh sibilance, loud breaths, cut-off endings, loudness too high or low. Also estimates BPM, key and vocal range, transcribes lyrics and speech with whisper.cpp, and describes a song the way a listener would: sections (intro, verses, chorus, breaks), repeats,...

  • 5 stars
  • 0 votes
  • 0 copies
  • 0 views
  • Added November 7, 2026
ai-agentspythonrustgobashreactexpressgit

Works with

  • cli

Security analysis

A96/100
  • mediumInstalls packages at runtime which could introduce malicious dependencies

Pro scans all 10 files and shows the line behind each finding

Scanned November 7, 2026

npx -y skills add rlawhddja1231-hash/sound-ear --skill sound-ear --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Sound Ear?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Sound Ear
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/rlawhddja1231-hash-sound-ear/badge)](https://www.skillsdirectory.com/skills/rlawhddja1231-hash-sound-ear)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: sound-ear
description: "Lets Claude 'hear' audio files. Turns songs, vocals or video soundtracks into numbers and pictures with a bundled script, then finds problems with timestamps: clicks, clipping, abrupt cuts, pitch glitches, off-pitch notes, wobbly notes, harsh sibilance, loud breaths, cut-off endings, loudness too high or low. Also estimates BPM, key and vocal range, transcribes lyrics and speech with whisper.cpp, and describes a song the way a listener would: sections (intro, verses, chorus, breaks), repeats, drums, bass and named instruments, genre, loudness changes, mood and repeated lyric hooks. Built for checking vocal-synth songs (SynthV, VOCALOID, UTAU). Use whenever the user shares an audio or video file (wav, mp3, flac, m4a, ogg, opus, webm, mp4) or asks to listen, describe a song, get lyrics or subtitles, check a mix, find glitches, measure BPM/key/loudness, compare two versions, or compare the singer's lyrics against what was actually sung (가사 대조: finds mispronounced or garbled syllables in vocal-synth songs). With the synth project file (.svp/.vpr/.ust/.mid) it reports problems per note number (노트 검사). Also handles hour-long files and writes a self-contained HTML report with audio snippets. Voice check (v2.0): scores how natural an AI voice, TTS, voice-over or dubbing sounds (1-5, UTMOS naturalness), marks the seconds that sound off, auto-separates background music first, flags long gaps, flat intonation, speaking rate and music drowning the voice, compares the script with what was said, and picks the best of several takes (ElevenLabs, OpenAI TTS, Typecast, CapCut, Synthesizer V talk, etc.). 소리 듣기: 노래 들어 봐, 검수, 가사, 받아쓰기, 자막, 곡 구성, 후렴, 분위기, 음정, 음 튐, 딸깍, 잡음, 깨짐, 끊김, BPM, 키, 음역, 믹싱, 두 판 비교, 가사 대조, 발음, 뭉개짐, 긴 파일, HTML 보고서, 노트 검사, 몇 번째 노트, svp, 브이신스 프로젝트, 출처 검사, 목소리 검사, AI 목소리, TTS, 더빙, 내레이션, 자연스러워?, 기계 같아, 대본 대조, 여러 판 중 골라, 쇼츠 목소리."
license: MIT
---

# sound-ear — listen to audio and find problems

Claude has no ears. This skill turns an audio file into **numbers and pictures** with `scripts/listen.py`, so Claude can inspect it, point at exact timestamps, and explain how to fix each problem. Reply to the user in their own language (the report and chart labels are in Korean; translate them when the user writes in another language).

What it cannot do: listen live through a microphone or judge taste. Lyrics need a whisper model file; instrument, genre and mood names need Essentia model files (see setup). Pitch checks need a **vocal-only** file; for a full mix, separate it first (see 2b).

## 1. Setup (once per session)

```bash
L="${CLAUDE_SKILL_DIR:-}/scripts/listen.py"
[ -f "$L" ] || L=$(find / -name listen.py -path '*sound-ear*' 2>/dev/null | head -1)
pip install -q librosa soundfile pyloudnorm matplotlib koreanize-matplotlib scipy 2>/dev/null \
  || pip install -q --break-system-packages librosa soundfile pyloudnorm matplotlib koreanize-matplotlib scipy
python3 "$L" setup          # checks/installs everything, fetches reachable models; --check = status only
```

`setup` installs the optional packages, gathers model files the user attached (from the working folder, `/mnt/user-data/uploads`, `~/Downloads`, or `--from DIR`; `X.bin.part1`+`.part2` are joined, an `essentia*.tar` is unpacked), downloads what it can reach, and prints a ✅/❌ table plus the URL of every model it could not fetch. If something is missing, show the user that list in plain words and ask them to attach the file, then run `setup` again. Add `--accurate` to also fetch the 539 MB medium lyrics model. If ffmpeg/ffprobe are missing, `setup` downloads a static build for the OS (Windows: BtbN zip from GitHub, Linux: BtbN tar.xz, macOS: evermeet) into `~/.cache/sound-ear/ffmpeg/` and every later run uses it automatically; if that download is blocked, it prints the URL and the folder to drop `ffmpeg`/`ffprobe` into. Manual details follow.

**Lyrics / speech (optional):** `pip install pywhispercpp` and a whisper.cpp model file `ggml-*.bin` from https://huggingface.co/ggerganov/whisper.cpp (`ggml-small-q5_1.bin`, 190 MB, fast; or `ggml-medium-q5_0.bin`, 539 MB, much better for Korean sung lyrics, ~1.6× real time on 2 cores — the largest model found is used unless `--fast`). Put it in `~/.cache/sound-ear/` or pass `--model`, or set `SOUND_EAR_MODEL`. If the sandbox cannot reach Hugging Face, ask the user to download the file and attach it. Without a model, `describe` still works but guesses where the singing is from loudness only. With `--lang auto` the language is detected from the three densest 30-second stretches of singing (whisper.cpp alone looks only at the first 30 s, so long intros gave the wrong language). Vocal stems and vocal-only files are transcribed in pieces of up to 25 s split at pauses — fed whole, a stem often came back as one made-up line; full mixes are still transcribed whole, which worked better there.

**Instruments, genre and mood (optional):** `pip install essentia-tensorflow` and these files from https://essentia.upf.edu/models/ (about 26 MB; **CC BY-NC-ND 4.0, non-commercial** — never bundle or redistribute them, the user downloads them): `feature-extractors/discogs-effnet/discogs-effnet-bs64-1.pb`, and from `classification-heads/`: `mtg_jamendo_instrument-discogs-effnet-1.pb/.json` (40 instruments), `genre_discogs400-discogs-effnet-1.pb/.json` (400 genres), `mtg_jamendo_moodtheme-discogs-effnet-1.pb/.json` (56 moods). Put them in `~/.cache/sound-ear/essentia/` or pass `--essentia DIR` / set `SOUND_EAR_ESSENTIA`. About 6 s for a 4-minute song.

**Vocal separation (optional, recommended for full mixes):** `pip install onnxruntime` and the UVR MDX-Net model `Kim_Vocal_2.onnx` (67 MB) from the Ultimate Vocal Remover model repo: https://github.com/TRvlvr/model_repo/releases/download/all_public_uvr_models/Kim_Vocal_2.onnx → `~/.cache/sound-ear/` (or `--sep-model`, `SOUND_EAR_SEP_MODEL`). Not bundled; download it. Takes about as long as the song on 2 CPU cores.

Self-tests (optional): `python3 "$(dirname "$L")/../tests/selftest.py"` (~20 s; 7 planted problems found, none extra) and `python3 "$(dirname "$L")/../tests/mixtest.py"` (~10 s; synthetic song with drumless intro, verses and a repeated louder chorus — checks sections, drums and chorus matching). Both should print `PASS`. `tests/voicetest.py` (~15 s) checks the voice check: long pause, flat vs varied intonation, background detection, a changed script word, and that the score model runs.

## 1b. Fastest path: one command

```bash
python3 "$L" full song.mp3 --lang ko --out song_full          # quick (default): ~3–4 min for a 4-min song
python3 "$L" full song.mp3 --lang ko --out song_full --deep   # detailed: separation + medium model + vocal pitch
python3 "$L" full song.mp3 --lang ko --out song_full --lyrics 가사.txt   # + compare the written lyrics with what was sung
```

`full` prints an estimated total time first, then `[2/4] 흠 찾기 — 지금까지 13초, 남은 시간 약 1분` at each step and `받아쓰기 35% (1:24/4:02)` inside long steps. The same line is written to `<out>/진행.txt`, so when you run `full` in the background you can read that file to tell the user how far it is. **Files over 20 minutes** (podcasts, streams, long mixes) are cut into 10-minute pieces automatically (`_조각/`), each piece is checked, and one merged report comes out with absolute timestamps (`--no-split` turns this off). Every `full` and `check` also writes **`보고서.html`**: one self-contained page with the verdict, the table of findings with a ▶ player for each spot (2.4-second mp3 snippets embedded), the lyrics comparison, the song flow, and the pictures — give it to the user when they want to listen through the findings without opening folders.

Default `full` is **quick**: check the mix → describe with the small lyrics model, no separation. Use it unless the user asks for a detailed listen, lyrics accuracy or vocal pitch on a full mix; then run `--deep` (= separate → check the mix → pitch-check the separated vocal, key taken from the backing → describe with the largest model). **Report the result to the user as soon as it is ready; do not start extra runs they did not ask for.** Both write **`종합보고서.md`**, whose first line is a one-sentence verdict (e.g. "꼭 고칠 곳은 없음 👍 — 들어 보고 판단할 곳 3곳만…"); lead with it. 🟠 "들어 보고 판단" items show up even in professional songs (intentional effects, hi-hats), so say that rather than alarming the user. The report has a one-page summary (mood, key, tempo, LUFS, genre/mood/instruments, vocal range, structure line) plus one table of everything to fix, most urgent first, with where it was found (mix or vocal). Read that first, then open the detailed reports it links. `--deep` on a 4.5-minute song takes ~10 min on 2 cores (~6 min with `--fast`). Example output: `docs/example-report/` in the GitHub repo. A vocal-only file is detected automatically (almost nothing below 70 Hz, or vocal/보컬 in the name) and gets the pitch check even in the quick run; force it with `--mode vocal`; with `--deep`, a file with almost no backing is detected and checked as a vocal directly.

## 2. Workflow (step by step)

1. **Get the file.** Work on a copy; never edit the user's original.
2. **Pick the mode.** For vocal-synth songs, ask for both files if available:
   - vocal-only export → `--mode vocal` (pitch, glitches, sibilance, breaths)
   - full mix → `--mode mix` (clipping, loudness, clicks, cut ends)
   - `--mode auto` (default) picks vocal when the filename contains vocal/vox/voice/보컬.
3. **Run the check.**
   ```bash
   python3 "$L" check vocal.wav --mode vocal --out vocal_check
   python3 "$L" check mix.wav   --mode mix   --out mix_check
   ```
   About 30 s for a 3-minute vocal, 15 s for a 3-minute mix. Output: `보고서.md` (report), `result.json`, `00_전체.png` (overview), 30-second pages, and `확대/` close-ups for each problem.
4. **Look at the pictures yourself** (open the PNGs) — don't trust the report alone. Red dashed line = must fix, orange = listen and decide, gray = info. In the pitch row, dark green dots = smoothed pitch, light green = fine-grained pitch; a single light-green dot far from the rest is a pitch glitch. Drop findings that are clearly intentional (grace notes, deliberate breaths, vibrato, stutter/gate effects, hi-hats in a mix).
5. **Report** as a table: time | what is wrong | urgency | how to fix. Then share a few close-ups and short audio snippets so the user can hear each spot:
   ```bash
   python3 "$L" cut vocal.wav 1:23.4 --len 2 --mode vocal --out snippets
   ```
6. **After the user fixes it,** re-run `check` and compare versions:
   ```bash
   python3 "$L" compare old.wav new.wav --out compare
   ```
   `compare` runs `check` on both versions (reusing results), lines the two up even when the new version starts earlier or later (1-second loudness curves, up to ±30 s), and writes `비교.md` with a one-line verdict and three lists: **없어진 흠** (fixed), **새로 생긴 흠** (regressions — read these first), **그대로인 흠** (still there, with old → new timestamps), plus the numbers side by side. Lead with the verdict and the regressions.

## 2b. Hearing the song as a listener

```bash
python3 "$L" describe song.mp3 --out song_described          # transcribes too if a model is found
python3 "$L" words song.mp3 --lang ko --out song_lyrics      # lyrics/speech only; also writes 자막.srt
python3 "$L" describe song.mp3 --words song_lyrics/words.json # reuse an earlier transcription
```

With the separation model, `describe` first splits the song into `보컬.wav` and `반주.wav` in the output folder (reused on later runs), transcribes the vocal stem, fills gaps from the full mix (lines marked "원곡에서 받아씀"), and uses the vocal stem to find where singing is. You can also run the pitch check on a full mix this way:

```bash
python3 "$L" separate song.mp3 --out stems            # stems/보컬.wav, stems/반주.wav
python3 "$L" check stems/보컬.wav --mode vocal        # key is taken from stems/반주.wav automatically
```

`describe` writes `곡설명.md`, `describe.json` and `곡흐름.png`: a one-line mood guess, key and tempo, a timeline table (section, repeat letter, loudness bar, drums, bass, singing, lyrics), a plain-language "as heard" walkthrough, repeated lyric hooks, and the full lyrics. Open the picture before you speak.

How it decides (so you can explain or overrule it):
- **Sections**: novelty curve on beat-synchronous chroma + timbre + loudness; repeats are long diagonal stripes in a chroma recurrence matrix. Same letter = same melody/harmony.
- **Chorus**: a repeated section that *returns after other parts* (back-to-back repeats count once) and whose repeats share lyrics — either overall or at least one matching line (a hook). Same melody with different lyrics is a verse. Without lyrics it falls back to "repeated and clearly louder"; if neither holds it does not name a chorus.
- **Sections** also react to drums entering or leaving (percussive level is part of the novelty features), and short songs get at least 5 parts.
- **Drums**: percussive part (HPSS) hitting regularly on the beat (autocorrelation ≥ 0.45). Vocal-only tracks score ~0.25.
- **Mood**: the one-line guess comes from key (major/minor), tempo, loudness and brightness. With Essentia models, the music AI's genre, mood and instrument guesses (with confidence %) are added: above ~30 % is fairly sure, ~15 % is "maybe". Per-section instruments show what enters or leaves.
- **Vocal-only files**: the instrument AI was trained on full mixes and hears drums in a cappella tracks; when the beat-regularity check finds no drums anywhere, drum-like labels are dropped, and when there is also no bass anywhere, instrument names are skipped.
- **Lyrics**: whisper.cpp; lines that look made up during instrumentals (subtitle credits, "please subscribe", bracketed song titles, stretched syllables, repeated lines) are flagged or dropped. With separation, the vocal stem is transcribed and only gaps where the voice is audible but no words came out are re-transcribed from the mix ("원곡에서 받아씀"). Sung lyrics have mistakes, especially over loud backing; vocal-only files are much better. About 25–40 s per minute of audio on 2 CPU cores with the small model.

## 2c. Lyrics comparison (내가 쓴 가사 ↔ 들린 것)

For vocal-synth songs the user usually has the exact lyrics they typed into the synth. Ask for them as a text file (one line per phrase; `[Verse]`-style bracket lines are ignored) and run:

```bash
python3 "$L" lyrics vocal.wav 가사.txt --out song_lyrics_check     # transcribes (uses the first lines as a hint), then compares
python3 "$L" lyrics song.mp3 가사.txt --words song_full/3_곡설명/words.json   # reuse an earlier transcription
python3 "$L" full song.mp3 --lyrics 가사.txt                       # same, folded into the summary report (section 4_가사대조)
```

How it works: both texts are stripped of spaces and punctuation, Korean syllables are split into jamo (초성·중성·종성), Japanese kanji/katakana become hiragana when `pykakasi` is installed, and the two character streams are aligned with a longest-common-subsequence match. A syllable with any differing jamo is marked 【】 (red in the HTML). Each written line gets a syllable score and a time range (from the aligned transcript pieces); lines with a differing syllable are 🟠, lines under 60 % are 🔴, and the 🔴/🟠 lines are cut to `가사조각/*.wav` so the user can listen. Transcript pieces that match almost nothing are listed as "가사에 없는데 들린 말" (ad-libs, backing vocals, or hallucinations).

How to talk about it: the transcription itself makes mistakes, so **a marked syllable is a place to listen, not a verdict**. Tell the user: if the same syllable is heard correctly elsewhere in the song, the problem is probably that spot (phoneme length, consonant strength, note too short, wrong phoneme entered); if a sound is wrong everywhere (ㅅ heard as ㅆ, dropped ㄹ batchim, 君 read as くん), it is the model's habit. Overall match under ~45 % means the lyrics file does not belong to this audio, the language is wrong, or the backing is drowning the voice — rerun with `--deep` or a vocal-only export. Use `--lang` when auto-detect picks the wrong language.

## 2e. Which note is wrong? (노트 검사 — project file + rendered vocal)

```bash
python3 "$L" notes vocal.wav song.svp --out song_notes          # Synthesizer V .svp, VOCALOID .vpr, UTAU .ust, MIDI .mid
python3 "$L" notes vocal.wav song.svp --track 보컬 --offset 0.5  # pick one track; force the audio/project time shift
```

When the user has the project file, ask for it: the report then names **note numbers** instead of timestamps. The script reads notes (start, length, pitch, lyric) from the project, tracks pitch on the rendered vocal, finds the time shift between audio and project automatically (±1.5 s; renders often start with silence), and writes `노트표.md` + `notes.json` + `노트그림.png` (grey/red note bars with the heard pitch in green). Per note it reports: median pitch offset in cents (🟠 over ±30, 🔴 over ±60 with the note it actually sounds like), notes that are silent in the render (🔴), notes that sound much shorter than written (end dies early / short phoneme), written length under 80 ms with a syllable (🟠 "발음이 뭉개질 수 있음"), wide pitch swing inside a long note, and any click / cut / glitch from the audio check mapped onto the note (start / middle / end). The header gives the global tuning offset — if most notes lean the same way, say so first (reference pitch or backing mismatch), not note by note. Self-test: `tests/notestest.py` (planted: +45 cents, 50 ms note, silent note, short-sounding note, mid-note tick → all found, `PASS`). Vocal-only renders only; for a full mix, separate first (2b). Lead with the 🔴 notes and give the fix in the synth's own terms (pitch deviation, note length, phoneme timing, note-end fade).

## 2d. Where did this file come from? (출처 검사)

```bash
python3 "$L" origin song.mp3 --out song_origin     # also runs inside `full` → 0_출처/출처.md and one line in the summary
```

Reads every metadata tag (ffprobe), scans the file bytes for a C2PA provenance manifest (JUMBF) and for generator names (Suno, Udio, Mureka, Riffusion, Stable Audio, ElevenLabs, MusicGen, Synthesizer V, VOCALOID…), and checks the file name for a Suno/Udio-style UUID. Verdicts: **출처 확인됨** (a tag, URL or C2PA signature names the maker — reliable), **흔적은 있지만 확실하진 않음** (prompt-like comment, maker name somewhere in the bytes, UUID file name — go and check: open `suno.com/song/<id>` in the browser, or Google one distinctive transcribed lyric line in quotes), **흔적 없음** (tags stripped or never written — say "unknown", never "human-made"). Tags and signatures are easy to strip, so absence proves nothing. There is deliberately **no audio-only "is this AI?" score**: on 2026-10-26 the long-term spectra of 4 public Suno songs and 3 human recordings overlapped completely (see references/log.md), and a trained classifier would mislabel human songs and break with every model update. If the user asks for one, explain that and offer the origin check plus the browser look-up instead.

## 2f. AI voice / TTS / voice-over check (목소리 검사, v2.0)

```bash
python3 "$L" voice narration.mp3 --out narr_voice                       # naturalness score + spots that sound off + speech habits
python3 "$L" voice narration.mp3 --script script.txt --out narr_voice   # + compare the script with what was said
python3 "$L" pick take1.wav take2.wav take3.wav --script script.txt --out takes   # rank several takes of the same script
```

Use it for any spoken audio: TTS (ElevenLabs, OpenAI, Typecast, CapCut…), AI dubbing, a narrated short, a voice clone, a podcast read. It writes `목소리검사.md`, `voice.json`, `목소리_전체.png` and `보고서.html` (pick: `고르기.md` / `고르기.html` plus one folder per take).

What it measures:
- **Naturalness 1–5** with UTMOS22 strong (SpeechMOS port, MIT; weights 411 MB from GitHub, fetched by `setup` into `~/.cache/sound-ear/`; needs `torch`). It imitates human MOS ratings: recorded human speech ~4, good modern TTS 3.5–4.3, old robotic TTS ~2. Bands: ≥4.0 사람처럼 자연스러움, ≥3.5 괜찮음, ≥3.0 기계 티가 남, ≥2.3 어색함, below that 많이 어색함. A **curve** of 4-second windows (1 s step) shows where it dips; dips ≥0.6 below the file's score (and under 3.2) become "어색하게 들림" with a time range — tell the user to regenerate just that sentence.
- **Background music is separated first, automatically.** If the quietest 2 % and loudest 5 % of the file are less than 40 dB apart, something is playing under the voice (TTS and clean voice measured 48–82 dB, music-backed shorts 14–35 dB; the 10 % point misread a TTS with few pauses as music); the score model punishes any music (on 2026-11-02 seven real Korean shorts scored 1.25–2.23 with music and 1.42–3.95 on the separated voice; a music-free file stayed 3.69→3.66). So with `Kim_Vocal_2.onnx` present the voice is split off (`분리/`), scored and transcribed alone; the mix score is kept as "배경음 포함 점수(참고)". It also reports how much louder the voice is than the music: under 6 dB → 🟠 music drowns the voice, 6–8 dB → info. `--separate no|yes` overrides.
- **Speech habits** (on the voice): talk ratio — speaking time ÷ span; popular shorts measured 70–95 %, under 55 % → 🟠 "빈 곳이 많음" (an AI-dubbed test short was 45 %: 31 s with only 13 s of talking — its voice scored 3.95, so the empty gaps, not the voice, made it feel flat); gaps ≥1.6 s → 🟠 "쉼이 김"; ≥12 s without any 0.2 s pause → info; intonation range per sentence (10–90 % pitch spread, octave errors removed; measured per sentence so two speakers don't inflate it) under 2.5 semitones → 🟠 monotone (real shorts measured 3.3–11.5); speaking rate in Korean syllables per second of actual speech (real shorts 5.8–6.5; over 9 → 🟠 too fast; under 3.5 → info).
- **Glitches** on the original file: clipping, loudness, clicks, cuts, sibilance (same checks as `check`). In speech, 3+ ticks within 0.5 s are a low voice creaking at phrase ends (vocal fry — each glottal pulse looks like a tick; one male TTS take showed 31 "clicks" that were all fry), so they are folded into one info line; an isolated tick inside voiced speech below 140 Hz is a glottal pulse of a low voice and is folded too; only the rest stay as 🟠.
- **The score cannot hear words.** A TTS that cannot speak Korean produced gibberish and still scored 3.97 (the best of six). Always pair the score with `--script`; under 60 % script match is 🔴 "대본과 많이 다름 — don't use this take".
- **Script comparison** (`--script`): sentences are split on . ? ! and aligned with the transcript at jamo level exactly like 2c, with numbers normalised to their Korean reading on both sides (10분의 1 = 십 분의 일, 5배 = 다섯 배, 72.4% = 칠십이점사퍼센트) and ㅐ/ㅔ treated as the same sound (앤트로픽 = 엔트로픽); the script is **not** passed to whisper as a hint (for speech it made whisper skip or repeat the opening lines), and the music filter's "made-up line" flag (구독, 좋아요…) is lifted because in speech those are real lines; changed or garbled words are 【marked】 with their time, words heard but not in the script are listed. Results also go to `대본대조.md`.
- **pick**: ranks takes by naturalness − 0.35 per must-fix (max 1.4) − 0.08 per listen-and-decide (max 0.6) − 2 × the share of the script that differs. Lines that every take reading the script well (≥80 %) gets "wrong" in the same way are whisper's habit, not the voice, so they are dropped from the script score and listed. Running `pick` again into the same `--out` reuses each take's `words.json` (~2 min instead of ~12 for six takes; `--fresh` re-transcribes). Field test 2026-11-02, one Korean news script × six Venice TTS takes: ElevenLabs turbo 3.25 / 100 % ranked first; MiniMax HD CasualGuy (two takes) 3.15 and 3.06 / 98 %, and only that voice read 하이쿠 as 하이코·하이크; Gemini 3.01 / 100 %; a take that stopped after 16 s and a voice that cannot speak Korean (scored 3.97!) both came last at 36–38 % script match. Whisper is loaded once and reused across takes (reloading it per take ran the sandbox out of memory on the sixth take). If the top two are within 0.1, say so and have the user listen to both.

How to talk about it: lead with the verdict line and the one or two things that matter most (separated score, talk ratio, the worst dip, script mismatches). Say plainly that the score model learned mostly from English, so absolute numbers for Korean are approximate and **comparing takes of the same script is the most reliable use**. A high score does not mean the delivery fits the scene — it measures how human and clean the voice sounds, not acting.

## 3. What each finding means and how to fix it

Menu names differ between vocal synths and versions; treat these as general directions.

| Finding (Korean label) | Meaning | Typical fix |
|---|---|---|
| 클리핑 (clipping) | Samples hit full scale; audible crackle | Lower track/master gain; limiter ceiling −1 dB |
| 소리가 0dB까지 꽉 참 | Lossy file (mp3/opus) touching 0 dB; can't prove real clipping | Check the original wav; master is very hot |
| 트루 피크 넘침 (true peak over) | Fine now, may distort after mp3/AAC encoding | Limiter ceiling −1 dBTP |
| 딸깍·틱 소리 (click) | Very short tick | Inspect note joins; re-render that part; add short fades at edit points |
| 노트 끝 틱 (note-end tick) | A loud note stops dead and ticks | Add a short fade / extend the note tail |
| 틱이 몰린 구간 (clustered ticks) | Many ticks within 10 s in a mix | Often a deliberate stutter/glitch edit; otherwise check edit seams |
| 음높이를 믿기 어려운 곳 | Harmony or noise; pitch checks skipped there | Check with a single-voice export |
| 뚝 끊김 (abrupt cut) | Sound stops instantly instead of decaying | Extend note tail / fade; check region ends (ignore if it's a gate effect) |
| 끝이 잘림 / 시작이 잘림 | Export range shorter than the sound | Extend export range; add fade-out |
| 음이 순간 튐 (pitch glitch) | Pitch jumps for tens of ms and returns | Remove the spike in the pitch curve; adjust note transition; check overlapping notes |
| 음정 어긋남 (off-pitch) | Note centre more than 30 cents off | Reset pitch deviation/offset near 0; reduce over-strong auto expression |
| 음 흔들림 큼 (wobbly) | Long note swings widely | Reduce vibrato depth or delay its onset |
| 조에 없는 음 (out of key) | Note not in the estimated key | Ignore if intentional |
| 전체 튜닝 치우침 | Whole vocal off A=440 | Match the reference pitch of vocal and backing |
| ㅅ·ㅊ 소리 셈 (sibilance) | "s/ts" as loud as vowels | Lower consonant volume/length; de-esser |
| 숨소리 큼 (loud breath) | Loud breath/noise between phrases | Fine if intentional; otherwise lower it |
| 너무 큼 / 너무 작음 | Integrated loudness far from −14 LUFS | Streaming services normalize; very loud masters just get turned down and sound squashed |
| 위상 반대 (phase) | L/R cancel on mono phone speakers | Flip polarity of one channel |

## 4. Detection thresholds (inside the script)

- Clipping: ≥0.999 for more than 3 samples in a row (lossy files: summary only)
- Click: >3 kHz energy in a 1 ms bin over 8× the 150 ms local median, with 3–8 ms neighbours under a third of it, skipping note/drum onsets where the level jumps more than 1.5× (in mix mode downgraded to "listen and decide")
- Abrupt cut: from above −25 dB to below −60 dB within 10 ms
- Pitch glitch: fine pitch (yin, 5.8 ms hop) more than 3 semitones from the local median and back within 80 ms, with an abrupt step, at ≥60 % of phrase level, confirmed by the spectrum (new fundamental present, old one gone, no octave confusion); skipped where pitch is unreliable (harmony/noise clusters)
- Off-pitch: notes ≥0.25 s whose median is >30 cents off after removing the global tuning offset
- Sibilance: 5–10 kHz energy within 6 dB of vowel level (75th percentile), >50 % of the frame, ≥30 ms

Tuned on synthetic tests, two vocal-only synth tracks and three full mixes. If you see false positives or misses on real material, note them in `references/log.md` with the date and adjust.

Files in this skill

  • SKILL.md28.5 KB
  • references/log.md14.3 KB
  • scripts/utmos/LICENSE-SpeechMOS1 KB
  • scripts/utmos/fairseq_alt.py8.7 KB
  • scripts/utmos/model.py2.3 KB
  • tests/make_test.py1.9 KB
  • tests/mixtest.py5.1 KB
  • tests/notestest.py3.3 KB
  • tests/selftest.py1.5 KB
  • tests/voicetest.py4.3 KB

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.