Libri2mix Noisy Eval

---
name: libri2mix-noisy-eval
description: Evaluates the ability of end-to-end speech separation models to isolate target speakers from noisy multi-speaker mixtures. It probes noise-robustness and speaker separation capability under realistic background noise conditions. Use when the user wants to benchmark on Libri2Mix-noisy, Libri3Mix-noisy, or asks about evaluating this task. Reports SI-SNRi (dB).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2302.11131
  bibtex_key: hu2023unifying
  confidence: high
---

# libri2mix-noisy-eval

> Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation — Hu et al. (2023) (arXiv:2302.11131, 2023)

## What this evaluates

Evaluates the ability of end-to-end speech separation models to isolate target speakers from noisy multi-speaker mixtures. It probes noise-robustness and speaker separation capability under realistic background noise conditions.

## Datasets

- **Libri2Mix-noisy** — total ?; splits: train-360 (-1), train-100 (-1), dev (-1), test (-1); repo https://github.com/JorisCos/LibriMix
- **Libri3Mix-noisy** — total ?; splits: train-360 (-1), train-100 (-1), dev (-1), test (-1); repo https://github.com/JorisCos/LibriMix

## Metrics

- `SI-SNRi (dB)` **(primary)** — range: dB
  - SI-SNRi = SI-SNR(estimated, clean) - SI-SNR(mixture, clean). Measures the improvement in signal-to-noise ratio over the noisy mixture input.
- `SDRi (dB)` — range: dB
  - SDRi = SDR(estimated, clean) - SDR(mixture, clean). Measures the improvement in signal-to-distortion ratio over the noisy mixture input.

## Input / output format

**Input**: 8 kHz sampled noisy speech mixtures containing 2 or 3 speakers, with background noise added at a mean SNR of -2 dB.

**Output**: Separated 8 kHz speech waveforms for each target speaker.

## Scoring recipe

```python
def compute_si_snr_i(est, clean, mix):
    # Align estimated signal to clean signal (optimal scaling & delay)
    est_aligned = align_and_scale(est, clean)
    si_snr_est = si_snr(est_aligned, clean)
    si_snr_mix = si_snr(mix, clean)
    return si_snr_est - si_snr_mix
```

## Common pitfalls

- SI-SNRi reports improvement over the noisy mixture, not absolute separation quality.
- Noise is added at ~-2 dB mean SNR with 3.6 dB std dev; results are not comparable to clean Libri2Mix benchmarks.
- Models must be evaluated on the test split (11 h), not the dev split, to match reported SOTA comparisons.

## Evidence (verbatim from paper)

> We conduct experiments on the large-scale benchmark Libri2Mix and Libri3Mix datasets (noisy version) to evaluate our proposed approach... Our best system achieves the state-of-the-art with a SI-SNR improvement (SI-SNRi) of 16.0 dB and a Signal-to-Distortion Ratio improvement (SDRi) of 16.5 dB.

## Citation

```bibtex
@misc{hu2023unifying,
  title={Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation},
  author={Hu et al. (2023)},
  year={2023},
  note={arXiv:2302.11131}
}
```

- arXiv: 2302.11131