Insectset459 Eval

---
name: insectset459-eval
description: Evaluates multi-class bioacoustic classification of insect audio recordings into one of 459 species. It probes a model's robustness to severe class imbalance, highly variable sampling rates, and ultrasonic frequency ranges. Use when the user wants to benchmark on InsectSet459, or asks about evaluating this task. Reports F1 score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2503.15074
  bibtex_key: faiss2025insectset459
  confidence: high
---

# insectset459-eval

> InsectSet459: an open dataset of insect sounds for bioacoustic machine learning — Faiß et al. (2025) (arXiv:2503.15074, 2025)

## What this evaluates

Evaluates multi-class bioacoustic classification of insect audio recordings into one of 459 species. It probes a model's robustness to severe class imbalance, highly variable sampling rates, and ultrasonic frequency ranges.

## Datasets

- **InsectSet459** — total 26399; splits: train (15873), validation (5307), test (5219)

## Metrics

- `F1 score` **(primary)** — range: percent
  - Macro-averaged F1 score computed across all 459 species classes.
- `accuracy` — range: percent
  - Per-item classification accuracy, calculated as the proportion of correctly predicted audio files.

## Input / output format

**Input**: Raw audio files (WAV or MP3) with highly variable sample rates (8–500 kHz) and durations (typically ~10s, trimmed to 120s max).

**Output**: Predicted species label from the 459-class taxonomy.

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels):
    accuracy = sum(p == g for p, g in zip(predictions, gold_labels)) / len(gold_labels)
    classes = set(gold_labels)
    f1_scores = []
    for cls in classes:
        tp = sum(p == cls and g == cls for p, g in zip(predictions, gold_labels))
        fp = sum(p == cls and g != cls for p, g in zip(predictions, gold_labels))
        fn = sum(p != cls and g == cls for p, g in zip(predictions, gold_labels))
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1_scores.append(f1)
    macro_f1 = sum(f1_scores) / len(f1_scores)
    return {'accuracy': accuracy, 'macro_f1': macro_f1}
```

## Common pitfalls

- Severe long-tail class imbalance means accuracy can be misleading; macro-F1 is required to assess rare species performance.
- Standard spectrogram features (e.g., up to 16–22 kHz) discard ultrasonic content, causing systematic underperformance for species calling outside this range.
- Variable sample rates across files require careful resampling or rate-aware processing to avoid spectral distortion or information loss.

## Evidence (verbatim from paper)

> Table 3. Overall classification performance of classifiers trained on IS459, evaluated on the test set. F1 score is macro-averaged across classes; accuracy is averaged per-item.

## Citation

```bibtex
@misc{faiss2025insectset459,
  title={InsectSet459: an open dataset of insect sounds for bioacoustic machine learning},
  author={Faiß et al. (2025)},
  year={2025},
  note={arXiv:2503.15074}
}
```

- arXiv: 2503.15074