Music Audio Tagging Eval

---
name: music-audio-tagging-eval
description: This evaluation probes a model's ability to perform large-scale music audio tagging by predicting a fixed set of semantic labels (e.g., genre, mood, instrumentation) from raw 30-second audio clips. It measures how well architectures generalize across varying dataset sizes and label granularities. Use when the user wants to benchmark on MagnaTagATune (MTT), Million Song Dataset (MSD), Private Dataset, or asks about evaluating this task. Reports top-50 tag prediction.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1711.02520
  bibtex_key: [REDACTED]
  confidence: medium
---

# music-audio-tagging-eval

> End-to-end learning for music audio tagging at scale — Jordi Pons et al. (2017) (arXiv:1711.02520, 2017)

## What this evaluates

This evaluation probes a model's ability to perform large-scale music audio tagging by predicting a fixed set of semantic labels (e.g., genre, mood, instrumentation) from raw 30-second audio clips. It measures how well architectures generalize across varying dataset sizes and label granularities.

## Datasets

- **MagnaTagATune (MTT)** — total 26000; splits: test (-1)
- **Million Song Dataset (MSD)** — total 240000; splits: test (-1)
- **Private Dataset** — total 1200000; splits: train (1000000), val (100000), test (100000)

## Metrics

- `top-50 tag prediction` **(primary)** — range: [0, 1]
  - Multi-label classification metric where the model outputs a ranked list of tags; performance is measured by how many of the ground-truth tags appear in the top-50 predictions.

## Input / output format

**Input**: 30-second music audio clips (waveforms or spectrograms)

**Output**: Ranked list of top-50 predicted tags per audio clip

## Scoring recipe

```python
def score(predictions, gold_tags):
    correct = 0
    for pred, gold in zip(predictions, gold_tags):
        if set(gold).issubset(set(pred)):
            correct += 1
    return correct / len(predictions)
```

## Common pitfalls

- Artist leakage: Splits are artist-filtered, so models must not memorize artist-specific styles to generalize.
- Dataset scale disparity: Performance trends differ drastically between ~26k samples (MTT) and ~1.2M samples (private), requiring careful scaling analysis.
- Tag granularity: The private dataset uses 139 fine-grained tags (e.g., subgenres, rhythmic feel), which are harder to predict than standard genre tags.

## Evidence (verbatim from paper)

> Predicting the top-50 tags of this dataset is a popular benchmark for auto-tagging.

## Citation

```bibtex
@misc{pons2017endtoend,
  title={End-to-end learning for music audio tagging at scale},
  author={Jordi Pons et al. (2017)},
  year={2017},
  note={arXiv:1711.02520}
}
```

- arXiv: 1711.02520