Fma Genre Classification Eval

---
name: fma-genre-classification-eval
description: Evaluates music information retrieval models on genre classification tasks using a large-scale, open music dataset. It probes the model's ability to map audio tracks to hierarchical genre labels (single-label or multi-label) using raw audio or precomputed features. Use when the user wants to benchmark on FMA (Free Music Archive), or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1612.01840
  bibtex_key: [REDACTED]
  confidence: high
---

# fma-genre-classification-eval

> FMA: A Dataset For Music Analysis — Defferrard et al. (2016) (arXiv:1612.01840, 2016)

## What this evaluates

Evaluates music information retrieval models on genre classification tasks using a large-scale, open music dataset. It probes the model's ability to map audio tracks to hierarchical genre labels (single-label or multi-label) using raw audio or precomputed features.

## Datasets

- **FMA (Free Music Archive)** — total 106574; splits: train (-1), val (-1), test (-1); repo https://github.com/mdeff/fma

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Standard classification accuracy: the proportion of correctly predicted genre labels out of the total number of test instances.

## Input / output format

**Input**: 30-second MP3 audio clips (for Large/Medium/Small subsets) or full-length MP3s (for Full subset), optionally accompanied by precomputed librosa features (518 features) and metadata.

**Output**: A predicted genre label (or set of labels) from the 161-category hierarchical taxonomy.

## Scoring recipe

```python
def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)
```

## Common pitfalls

- Artist leakage: using tracks from the same artist in both train and test sets inflates accuracy.
- Genre hierarchy ambiguity: tracks can have multiple sub-genres; models must decide whether to predict top-level, specific, or all ancestor genres.
- Metadata noise: artist-provided genres may be inconsistent or motivated by play counts rather than objective classification.

## Evidence (verbatim from paper)

> It has been shown that the use of songs from the same artist in both training and test sets leads to over-optimistic accuracy and may favor some approaches

## Citation

```bibtex
@misc{defferrard2016fma,
  title={FMA: A Dataset For Music Analysis},
  author={Defferrard et al. (2016)},
  year={2016},
  note={arXiv:1612.01840}
}
```

- arXiv: 1612.01840