Dysarthric Asr Eval

---
name: dysarthric-asr-eval
description: Evaluates the ability of ASR and LLM-enhanced decoding models to accurately transcribe dysarthric speech across varying severity levels and domains. It probes robustness to phonetic distortions, grammatical consistency, and cross-dataset generalization. Use when the user wants to benchmark on TORGO, UASpeech, or asks about evaluating this task. Reports WER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.08027
  bibtex_key: aboeitta2025bridging
  confidence: high
---

# dysarthric-asr-eval

> Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches — Ahmed Aboeitta et al. (2025) (arXiv:2508.08027, 2025)

## What this evaluates

Evaluates the ability of ASR and LLM-enhanced decoding models to accurately transcribe dysarthric speech across varying severity levels and domains. It probes robustness to phonetic distortions, grammatical consistency, and cross-dataset generalization.

## Datasets

- **TORGO** — total ?; splits: test (-1)
- **UASpeech** — total ?; splits: test (-1)

## Metrics

- `WER` **(primary)** — range: [0, 1]
  - Word Error Rate, calculated as the normalized edit distance (substitutions, deletions, insertions) between the predicted and reference transcriptions, divided by the number of words in the reference.
- `CER` — range: [0, 1]
  - Character Error Rate, defined as CER = (S + D + I) / N, where S, D, and I are the number of substitutions, deletions, and insertions, and N is the number of characters in the reference transcription.

## Input / output format

**Input**: Dysarthric speech audio recordings.

**Output**: Text transcription of the spoken utterance.

## Scoring recipe

```python
def compute_metrics(predictions, references):
    wer_vals, cer_vals = [], []
    for pred, ref in zip(predictions, references):
        pred_words = pred.split()
        ref_words = ref.split()
        wer_vals.append(edit_distance(pred_words, ref_words) / len(ref_words) if ref_words else 0.0)
        cer_vals.append(edit_distance(list(pred), list(ref)) / len(ref) if ref else 0.0)
    return sum(wer_vals) / len(wer_vals), sum(cer_vals) / len(cer_vals)
```

## Common pitfalls

- Relying solely on WER may overlook semantic fluency improvements captured by qualitative analysis and CER.
- Cross-dataset generalization performance degrades significantly, so in-dataset results alone overestimate real-world robustness.

## Evidence (verbatim from paper)

> To assess transcription quality beyond WER, we analyze Character Error Rate (CER) and sample transcriptions, offering a finer evaluation of phoneme distortions and semantic accuracy.

## Citation

```bibtex
@misc{aboeitta2025bridging,
  title={Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches},
  author={Ahmed Aboeitta et al. (2025)},
  year={2025},
  note={arXiv:2508.08027}
}
```

- arXiv: 2508.08027