Low Resource Asr Eval

---
name: low-resource-asr-eval
description: Evaluates automatic speech recognition (ASR) performance on low-resource and high-resource languages using synthetic audio generated from text augmentation. It probes the model's ability to generalize to unseen lexical and syntactic variations when trained on limited real speech data. Use when the user wants to benchmark on Vatlongos, Nashta, Kakabe, Shinekhen Buryat, LibriSpeech, or asks about evaluating this task. Reports WER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.15373
  bibtex_key: ibaraki2025frustratingly
  confidence: high
---

# low-resource-asr-eval

> Frustratingly Easy Data Augmentation for Low-Resource ASR — Ibaraki et al. (2025) (arXiv:2509.15373, 2025)

## What this evaluates

Evaluates automatic speech recognition (ASR) performance on low-resource and high-resource languages using synthetic audio generated from text augmentation. It probes the model's ability to generalize to unseen lexical and syntactic variations when trained on limited real speech data.

## Datasets

- **Vatlongos** — total ?; splits: train (-1)
- **Nashta** — total ?; splits: train (-1)
- **Kakabe** — total ?; splits: train (-1)
- **Shinekhen Buryat** — total ?; splits: train (-1)
- **LibriSpeech** — total ?; splits: test-clean-54 (-1), test-clean-108 (-1), test-clean-324 (-1), train-clean-1207 (-1)

## Metrics

- `WER` **(primary)** — range: percent
  - Word Error Rate, calculated as the minimum number of insertions, deletions, and substitutions of words required to transform the predicted transcription into the reference transcription, divided by the total number of words in the reference.

## Input / output format

**Input**: Raw audio waveform (synthetic or original) fed into a Wav2Vec2-XLSR-53 encoder.

**Output**: CTC token sequence decoded into a text transcription.

## Scoring recipe

```python
def calculate_wer(predictions, references):
    total_errors = 0
    total_words = 0
    for pred, ref in zip(predictions, references):
        dist = edit_distance(pred.split(), ref.split())
        total_errors += dist
        total_words += len(ref.split())
    return (total_errors / total_words) * 100 if total_words > 0 else 0.0
```

## Common pitfalls

- Using a higher ratio of synthetic to original data than 1:1 degrades performance.
- Assuming semantic coherence is required; the augmentation deliberately uses hallucinated or ungrammatical text to test robustness.
- Voice cloning is unnecessary and was found to yield no performance improvements over standardized TTS voices.

## Evidence (verbatim from paper)

> achieving up to a 14.3% WER reduction in Nashta ASR and demonstrating broad applicability across low-resource and high-resource languages by enhancing model robustness through lexical and syntactic variation rather than semantic fidelity.

## Citation

```bibtex
@misc{ibaraki2025frustratingly,
  title={Frustratingly Easy Data Augmentation for Low-Resource ASR},
  author={Ibaraki et al. (2025)},
  year={2025},
  note={arXiv:2509.15373}
}
```

- arXiv: 2509.15373