Libritts Tts Eval

---
name: libritts-tts-eval
description: Evaluates the naturalness and quality of synthesized speech from text-to-speech models trained on the LibriTTS corpus. It probes how audio sampling rate, text normalization, and sentence-level splitting affect human-perceived speech naturalness compared to the original LibriSpeech dataset. Use when the user wants to benchmark on LibriTTS, or asks about evaluating this task. Reports MOS.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1904.02882
  bibtex_key: zen2019libritts
  confidence: high
---

# libritts-tts-eval

> LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech — Zen et al. (2019) (arXiv:1904.02882, 2019)

## What this evaluates

Evaluates the naturalness and quality of synthesized speech from text-to-speech models trained on the LibriTTS corpus. It probes how audio sampling rate, text normalization, and sentence-level splitting affect human-perceived speech naturalness compared to the original LibriSpeech dataset.

## Datasets

- **LibriTTS** — total ?; splits: train-clean (-1), test (-1)

## Metrics

- `MOS` **(primary)** — range: other
  - Average human rating on a 5-point Likert scale (1: Bad, 2: Poor, 3: Fair, 4: Good, 5: Excellent) in increments of 0.5, assessing the naturalness of synthesized speech stimuli.

## Input / output format

**Input**: Character sequences with punctuation marks from normalized text.

**Output**: Synthesized speech waveform (audio).

## Scoring recipe

```python
ratings = []
for stimulus in stimuli:
    rating = human_listener_rate(stimulus, scale=[1, 5], step=0.5, criterion="naturalness")
    ratings.append(rating)
mos = sum(ratings) / len(ratings)
return mos
```

## Common pitfalls

- Comparing LibriTTS (~245 hours) directly to LibriSpeech (~460 hours) without controlling for the difference in training data size.
- The evaluation setup shows a consistent performance gap between male and female speakers, indicating the model configuration is suboptimal for male voices.
- Down-sampling 24kHz synthesized audio to 16kHz is used to isolate sampling rate effects, which may confound naturalness ratings if listeners are sensitive to bandwidth.

## Evidence (verbatim from paper)

> Quantitative subjective evaluations relied on crowd-sourced mean opinion scores (MOS) rating the naturalness of the synthesized speech by native speakers using headphones. After listening to each stimulus, a subject was asked to rate the naturalness of the stimulus in a five-point Likert scale score (1: Bad, 2: Poor, 3: Fair, 4: Good, 5: Excellent) in increments of 0.5.

## Citation

```bibtex
@misc{zen2019libritts,
  title={LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech},
  author={Zen et al. (2019)},
  year={2019},
  note={arXiv:1904.02882}
}
```

- arXiv: 1904.02882