Blizzard Indian G2p Dur Eval

---
name: blizzard-indian-g2p-dur-eval
description: Evaluates DNN-based grapheme-to-phoneme and duration prediction models for three Indian languages (Hindi, Tamil, Telugu) using crowdsourced ASCII transliterated text. It measures the accuracy of predicted phoneme durations against forced-aligned ground truth to assess component quality for speech synthesis. Use when the user wants to benchmark on Blizzard Challenge 2015 (Hindi, Tamil, Telugu), or asks about evaluating this task. Reports RMSE (frames per phone).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1608.05374
  bibtex_key: ronanki2016dnn
  confidence: high
---

# blizzard-indian-g2p-dur-eval

> DNN-based Speech Synthesis for Indian Languages from ASCII text — Ronanki et al. (2016) (arXiv:1608.05374, 2016)

## What this evaluates

Evaluates DNN-based grapheme-to-phoneme and duration prediction models for three Indian languages (Hindi, Tamil, Telugu) using crowdsourced ASCII transliterated text. It measures the accuracy of predicted phoneme durations against forced-aligned ground truth to assess component quality for speech synthesis.

## Datasets

- **Blizzard Challenge 2015 (Hindi, Tamil, Telugu)** — total 5653; splits: train (-1), dev (-1), test (-1)

## Metrics

- `RMSE (frames per phone)` **(primary)** — range: other
  - Root Mean Square Error between predicted and forced-aligned phoneme durations. Calculated as the square root of the average of squared differences between predicted and ground truth duration values.

## Input / output format

**Input**: ASCII transliterated text, tokenized by whitespace and punctuation.

**Output**: Predicted phoneme durations in frames.

## Scoring recipe

```python
def compute_rmse(predictions, gold):
    if len(predictions) != len(gold):
        raise ValueError('Length mismatch')
    squared_diffs = [(p - g) ** 2 for p, g in zip(predictions, gold)]
    mse = sum(squared_diffs) / len(squared_diffs)
    return math.sqrt(mse)
```

## Common pitfalls

- Splits are stratified by annotator rather than randomly split, meaning different annotators produced the train, dev, and test data to prevent leakage.
- The train/dev/test split ratios are 92%/4%/4%, which deviates from the common 90/5/5 or 80/10/10 conventions and may cause confusion.

## Evidence (verbatim from paper)

> Table 4: RMSE (frames per phone) between predicted and forced-aligned durations.

## Citation

```bibtex
@misc{ronanki2016dnn,
  title={DNN-based Speech Synthesis for Indian Languages from ASCII text},
  author={Ronanki et al. (2016)},
  year={2016},
  note={arXiv:1608.05374}
}
```

- arXiv: 1608.05374