Few Shot Audio Classification Eval

---
name: few-shot-audio-classification-eval
description: This benchmark evaluates few-shot audio classification capability across diverse acoustic domains including environmental sounds, musical instruments, bird species, and speaker recognition. It measures how well a model generalizes to novel classes with only a few labeled examples per class using a prototypical network framework. Use when the user wants to benchmark on ESC-50, FSD2018, NSynth, BirdCLEF 2020, VoxCeleb1, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.10074
  bibtex_key: sgouropoulos2025prototypical
  confidence: high
---

# few-shot-audio-classification-eval

> Prototypical Contrastive Learning For Improved Few-Shot Audio Classification — Sgouropoulos et al. (2025) (arXiv:2509.10074, 2025)

## What this evaluates

This benchmark evaluates few-shot audio classification capability across diverse acoustic domains including environmental sounds, musical instruments, bird species, and speaker recognition. It measures how well a model generalizes to novel classes with only a few labeled examples per class using a prototypical network framework.

## Datasets

- **ESC-50** — total 2000; splits: test (-1), val (-1)
- **FSD2018** — total 11000; splits: test (-1), val (-1)
- **NSynth** — total 300000; splits: test (-1), val (-1)
- **BirdCLEF 2020** — total 80000; splits: test (-1), val (-1)
- **VoxCeleb1** — total 57737; splits: test (-1), val (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Average classification accuracy computed over 2,000 randomly sampled 5-way, 5-shot tasks from the test set. Results are averaged across 5 independent experimental runs, with 95% confidence intervals reported.

## Input / output format

**Input**: 16 kHz audio clips converted to mel spectrograms, segmented to 5 seconds if variable-length, and globally standardized. Each few-shot episode consists of a support set (5 classes × 5 samples) and a query set. Spectrograms are processed through a CRNN backbone with a self-attention mechanism to produce 256-dimensional embeddings.

**Output**: Class labels for query samples, determined by nearest-prototype classification in the embedding space.

## Scoring recipe

```python
episode_accuracies = []
for _ in range(2000):
    support, query = sample_episode(test_set, n_way=5, n_shot=5)
    prototypes = mean_embeddings(support)
    preds = nearest_prototype(query, prototypes)
    episode_accuracies.append(mean(preds == query.labels))
return mean(episode_accuracies)
```

## Common pitfalls

- The evaluation relies on 2,000 randomized few-shot episodes per dataset, so results are highly sensitive to the random seed and task sampling strategy.
- Hyperparameters for the angular loss (angle threshold alpha and anchor strategy) are tuned separately per dataset rather than using a single fixed setting.
- Variable-length audio is truncated or padded to 5-second segments, which may discard contextual information or introduce artifacts not present in the original recordings.

## Evidence (verbatim from paper)

> We evaluate the best performing model on the validation set over 2,000 randomly sampled 5-way, 5-shot tasks from the test set. We repeat each experiment five times and report the average accuracy and the 95% confidence interval.

## Citation

```bibtex
@misc{sgouropoulos2025prototypical,
  title={Prototypical Contrastive Learning For Improved Few-Shot Audio Classification},
  author={Sgouropoulos et al. (2025)},
  year={2025},
  note={arXiv:2509.10074}
}
```

- arXiv: 2509.10074