Wildscore Eval

---
name: wildscore-eval
description: This benchmark evaluates multimodal large language models' ability to perform multi-step, context-sensitive reasoning over symbolic musical notation. It probes capabilities in harmonic analysis, rhythmic interpretation, structural form recognition, and expressive markings through multiple-choice questions derived from real-world compositions and forum queries. Use when the user wants to benchmark on WildScore, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.04744
  bibtex_key: mundada2025wildscore
  confidence: high
---

# wildscore-eval

> WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning — Mundada et al. (2025) (arXiv:2509.04744, 2025)

## What this evaluates

This benchmark evaluates multimodal large language models' ability to perform multi-step, context-sensitive reasoning over symbolic musical notation. It probes capabilities in harmonic analysis, rhythmic interpretation, structural form recognition, and expressive markings through multiple-choice questions derived from real-world compositions and forum queries.

## Datasets

- **WildScore** — total ?; splits: test (-1); repo https://github.com/GaganVM/WildScore

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered multiple-choice questions. Calculated as the number of correct predictions divided by the total number of questions, multiplied by 100.

## Input / output format

**Input**: Per instance: a symbolic score image (optional) and a multiple-choice question text. Two evaluation modalities are used: image+text and text-only.

**Output**: A single selected answer choice from the provided multiple-choice options.

## Scoring recipe

```python
def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- Models may perform better in the text-only setting than in the image+text setting (e.g., InternVL, MiniCPM, LLaVA), indicating that visual context can sometimes introduce noise or alignment issues rather than aiding reasoning.
- Failures must be carefully disentangled into perception-based errors (misreading notation) versus reasoning-based errors (incorrect music-theory application), as models can correctly perceive symbols but still fail the question.
- The benchmark uses naturalized, user-generated questions where perception of the score is not always strictly necessary, meaning high accuracy might sometimes reflect textual priors rather than genuine multimodal reasoning.

## Evidence (verbatim from paper)

> Following standard practice in multimodal reasoning benchmarks*Yu et al. ([2023])*, we adopt accuracy as our primary metric, calculated as the percentage of correctly answered multiple-choice questions. Each question includes one correct answer, annotated based on human or language model preference as detailed in Section [3].

## Citation

```bibtex
@misc{mundada2025wildscore,
  title={WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning},
  author={Mundada et al. (2025)},
  year={2025},
  note={arXiv:2509.04744}
}
```

- arXiv: 2509.04744