Medical Asr Denoising Eval

---
name: medical-asr-denoising-eval
description: Evaluates the robustness of modern medical ASR models to various noise conditions and assesses whether speech enhancement preprocessing improves or degrades transcription accuracy. Use when the user wants to benchmark on Medical ASR Recordings (unspecified), or asks about evaluating this task. Reports semWER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.17562
  bibtex_key: [REDACTED]
  confidence: high
---

# medical-asr-denoising-eval

> When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems — Chondhekar et al. (2025) (arXiv:2512.17562, 2025)

## What this evaluates

Evaluates the robustness of modern medical ASR models to various noise conditions and assesses whether speech enhancement preprocessing improves or degrades transcription accuracy.

## Datasets

- **Medical ASR Recordings (unspecified)** — total ?; splits: test (-1)

## Metrics

- `semWER` **(primary)** — range: percent
  - Semantic Word Error Rate, measuring the percentage of words incorrectly transcribed based on semantic equivalence rather than exact character matching.
- `ΔsemWER` — range: percent
  - Change in semWER after denoising, calculated as semWER(denoised) - semWER(noisy). Positive values indicate performance degradation.

## Input / output format

**Input**: Audio recordings of medical speech, provided in original, noisy (background, short, Gaussian at varying SNR/amplitude), or denoised states.

**Output**: ASR model transcriptions (text).

## Scoring recipe

```python
def compute_semwer(audio, reference):
    pred = asr_model(audio)
    return compute_semantic_wer(pred, reference)

def compute_delta_semwer(noisy_audio, denoised_audio, reference):
    semwer_noisy = compute_semwer(noisy_audio, reference)
    semwer_denoised = compute_semwer(denoised_audio, reference)
    return semwer_denoised - semwer_noisy
```

## Common pitfalls

- Assuming speech enhancement universally improves ASR performance, whereas this study shows it consistently degrades semantic accuracy across all tested models.
- Failing to distinguish between exact character WER and semantic WER (semWER), which accounts for paraphrasing and clinical terminology variations.
- Ignoring that different noise types (background, short, Gaussian) and SNR levels cause vastly different degradation magnitudes, with Gaussian noise causing catastrophic collapse for some models.

## Evidence (verbatim from paper)

> Figure 4 shows the change in ASR performance resulting from the denoising process. It presents the ΔsemWER (De-noised − Noisy) for all models, directly comparing the error rates for noisy audio against the same audio after enhancement.

## Citation

```bibtex
@misc{chondhekar2025medicalasrdenoising,
  title={When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems},
  author={Chondhekar et al. (2025)},
  year={2025},
  note={arXiv:2512.17562}
}
```

- arXiv: 2512.17562