Musdb18 Separation Eval

---
name: musdb18-separation-eval
description: Probes a model's ability to isolate specific audio sources from a mixture using a provided query signal. It evaluates how well the model handles continuous latent-space conditioning and separates arbitrary or subclass instruments beyond standard training labels. Use when the user wants to benchmark on MUSDB18, or asks about evaluating this task. Reports SDR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1908.06593
  bibtex_key: lee2019audioquery
  confidence: high
---

# musdb18-separation-eval

> Audio query-based music source separation — Lee et al. (2019) (arXiv:1908.06593, 2019)

## What this evaluates

Probes a model's ability to isolate specific audio sources from a mixture using a provided query signal. It evaluates how well the model handles continuous latent-space conditioning and separates arbitrary or subclass instruments beyond standard training labels.

## Datasets

- **MUSDB18** — total 150; splits: train (100), test (50)

## Metrics

- `SDR` **(primary)** — range: dB
  - Signal-to-Distortion Ratio computed using the official museval package. It measures the ratio of the target source energy to the distortion energy (artifacts, noise, interference).

## Input / output format

**Input**: Mixture audio resampled to 22050Hz converted to a magnitude spectrogram (STFT, window 1024, 75% overlap), paired with a 3-second query audio segment encoding the target source.

**Output**: Separated source magnitude spectrogram, reconstructed to audio via Inverse STFT using the original mixture's phase.

## Scoring recipe

```python
import museval
import numpy as np
# Evaluate mixture against ground truth source using official package
results = museval.eval_mus(mixture_path, gt_source_path)
# Extract SDR scores for each track
sdr_scores = [res.bss['SDR'] for res in results]
# Report median SDR across the test set
median_sdr = np.median(sdr_scores)
```

## Common pitfalls

- The model reuses the mixture's phase for reconstruction rather than predicting it, which can artificially inflate SDR compared to full-phase models.
- Performance is highly sensitive to the query latent vector; using the class mean vector yields lower SDR than using ground-truth or retrieved vectors, so baselines must be clearly specified.
- Evaluation is performed on 3-second segments, but museval aggregates metrics over full tracks; ensure consistent handling of segment boundaries.

## Evidence (verbatim from paper)

> We evaluated our method on the test set of MUSDB18 using the official museval package111https://sigsep.github.io/sigsep-mus-eval which computes signal-to-distortion ratio (SDR) as a quantitative measurement.

## Citation

```bibtex
@misc{lee2019audioquery,
  title={Audio query-based music source separation},
  author={Lee et al. (2019)},
  year={2019},
  note={arXiv:1908.06593}
}
```

- arXiv: 1908.06593