Wsj0 Speech Separation Eval

---
name: wsj0-speech-separation-eval
description: This evaluation probes a model's ability to perform single-channel speech separation by learning discriminative time-frequency embeddings that group mixture components into distinct speaker clusters. It specifically tests generalization to unseen speakers and scaling to three-speaker mixtures without retraining. Use when the user wants to benchmark on WSJ0-based Speech Mixtures, or asks about evaluating this task. Reports SDR improvement (dB).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1508.04306
  bibtex_key: hershey2015deepclustering
  confidence: high
---

# wsj0-speech-separation-eval

> Deep clustering: Discriminative embeddings for segmentation and separation — Hershey et al. (2015) (arXiv:1508.04306, 2015)

## What this evaluates

This evaluation probes a model's ability to perform single-channel speech separation by learning discriminative time-frequency embeddings that group mixture components into distinct speaker clusters. It specifically tests generalization to unseen speakers and scaling to three-speaker mixtures without retraining.

## Datasets

- **WSJ0-based Speech Mixtures** — total ?; splits: train (-1), val (-1), test_closed (-1), test_open (-1)

## Metrics

- `SDR improvement (dB)` **(primary)** — range: dB
  - Signal-to-Distortion Ratio (SDR) improvement calculated as the difference between the SDR of the separated signal and the SDR of the original mixture, averaged over test utterances. Computed using the bss_eval toolbox.

## Input / output format

**Input**: Log short-time Fourier spectral magnitudes of the mixture speech, computed with a 32 ms window, 8 ms shift, and square root of Hann window. Input is segmented into 100-frame chunks.

**Output**: Embedding matrix V mapping each time-frequency bin to a speaker cluster dimension. Separated signals are reconstructed by applying time-frequency masks derived from clustering V.

## Scoring recipe

```python
def compute_sdr_improvement(mixture, separated, reference):
    sdr_mix = bss_eval_sdr(mixture, reference)
    sdr_sep = bss_eval_sdr(separated, reference)
    return np.mean(sdr_sep - sdr_mix)
```

## Common pitfalls

- Within-segment clustering suffers from permutation ambiguity; the paper reports oracle permutation results (minimizing L2 distance to reference) as an upper bound, which is not achievable in practice without ground truth.
- Evaluation distinguishes between 'closed speaker' (speakers seen during training) and 'open speaker' (unseen speakers); methods often require speaker adaptation to perform on the open set.
- Training masks out time-frequency bins where source magnitude is below -40 dB of the max, which can bias the model if the evaluation protocol does not account for silent regions similarly.

## Evidence (verbatim from paper)

> For all the experiment, performance was evaluated in terms of averaged signal-to-distortion ratio (SDR) using the bss_eval toolbox [27]. The initial SDR averaged over the mixtures was 0.16 dB for two speaker mixtures and -2.95 dB for three speaker mixtures.

## Citation

```bibtex
@misc{hershey2015deepclustering,
  title={Deep clustering: Discriminative embeddings for segmentation and separation},
  author={Hershey et al. (2015)},
  year={2015},
  note={arXiv:1508.04306}
}
```

- arXiv: 1508.04306