Ears Reverb Eval

---
name: ears-reverb-eval
description: Evaluates dereverberation models by measuring their ability to remove room acoustics effects from speech using real room impulse responses with RT60 up to 2 seconds. The protocol ensures fair comparison by normalizing loudness and removing direct-path delays before convolution. Use when the user wants to benchmark on EARS-Reverb, or asks about evaluating this task. Reports SI-SDR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2406.06185
  bibtex_key: richter2024ears
  confidence: high
---

# ears-reverb-eval

> EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation — Richter et al. (2024) (arXiv:2406.06185, 2024)

## What this evaluates

Evaluates dereverberation models by measuring their ability to remove room acoustics effects from speech using real room impulse responses with RT60 up to 2 seconds. The protocol ensures fair comparison by normalizing loudness and removing direct-path delays before convolution.

## Datasets

- **EARS-Reverb** — total ?; splits: train (32485), val (632), test (886), blind_test (743)

## Metrics

- `SI-SDR` **(primary)** — range: dB
  - Scale-Invariant Signal-to-Distortion Ratio. Measures the ratio between the target signal energy and the residual noise/error energy, invariant to scaling.
- `POLQA` — range: [1, 5]
  - Perceptual Objective Listening Quality Analysis. ITU standard for predicting perceived speech quality based on psychoacoustic models.
- `PESQ` — range: [-0.5, 4.5]
  - Perceptual Evaluation of Speech Quality. ITU standard for predicting speech quality by comparing enhanced and reference signals.
- `ESTOI` — range: [0, 1]
  - Extended Short-Time Objective Intelligibility. Predicts speech intelligibility based on temporal envelope correlation in short-time segments.
- `SIGMOS` — range: [1, 5]
  - Speech Quality MOS. Deep learning-based predictor of mean opinion score for speech quality.
- `DNSMOS` — range: [1, 5]
  - Deep Noise Suppression MOS. Deep learning-based predictor of mean opinion score for speech quality and noise suppression.
- `WER` — range: percent
  - Word Error Rate. Percentage of words incorrectly recognized compared to the reference transcript.

## Input / output format

**Input**: Reverberant audio file (and clean reference audio).

**Output**: Dereverberated audio file.

## Scoring recipe

```python
def compute_metrics(predictions, references):
    scores = {}
    scores['SI-SDR'] = compute_si_sdr(predictions, references)  # dB
    scores['POLQA'] = compute_polqa(predictions, references)    # 1-5
    scores['PESQ'] = compute_pesq(predictions, references)      # -0.5 to 4.5
    scores['ESTOI'] = compute_estoi(predictions, references)    # 0-1
    scores['SIGMOS'] = compute_sigmos(predictions)              # 1-5
    scores['DNSMOS'] = compute_dnsmos(predictions)              # 1-5
    scores['WER'] = compute_wer(predictions, references)        # percent
    return scores
```

## Common pitfalls

- RIRs are truncated up to the highest amplitude index to remove direct-path delay before convolution.
- Loudness of reverberant speech is normalized to match clean speech using LKFS.
- Only RIRs with RT60 ≤ 2 s are used, and multi-channel recordings use a randomly selected channel.

## Evidence (verbatim from paper)

> Table 5: Results for the blind test. Column groups are the method name, intrusive metrics, non-intrusive metrics, and WER. Values indicate mean and standard deviation.

## Citation

```bibtex
@misc{richter2024ears,
  title={EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation},
  author={Richter et al. (2024)},
  year={2024},
  note={arXiv:2406.06185}
}
```

- arXiv: 2406.06185