Emid Emotional Alignment Eval

---
name: emid-emotional-alignment-eval
description: Human-subject validation of cross-modal emotional alignment between music and images. It probes whether pairing audio and visual stimuli based on a 13-dimensional emotional coordinate space yields higher perceptual matching accuracy compared to semantic-only or random matching. Use when the user wants to benchmark on EMID, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2308.07622
  bibtex_key: zou2023emid
  confidence: high
---

# emid-emotional-alignment-eval

> EMID: An Emotional Aligned Dataset in Audio-Visual Modality — Zou et al. (2023) (arXiv:2308.07622, 2023)

## What this evaluates

Human-subject validation of cross-modal emotional alignment between music and images. It probes whether pairing audio and visual stimuli based on a 13-dimensional emotional coordinate space yields higher perceptual matching accuracy compared to semantic-only or random matching.

## Datasets

- **EMID** — total ?; splits: test (91); repo https://github.com/ecnu-aigc/EMID

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Ratio of correct matches (subjects' answers matching the EMID's designated best pair) to the total number of validation questions. Calculated separately for music-to-image, image-to-music, and per emotional category.

## Input / output format

**Input**: Per trial: (1) Music-to-Image: One music clip. (2) Image-to-Music: Three images. Subjects select the best matching image or music clip from four options.

**Output**: Subject's selected option index (1–4) per trial.

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return correct / len(gold)
```

## Common pitfalls

- The evaluation uses a small subset (91 trials) rather than the full dataset, so results may not generalize to the entire 30k+ corpus.
- Category-level accuracy is skewed by dataset imbalance; for example, the 'disgusting' category was removed from the source image dataset, affecting category C results.
- The 'correct' answer is pre-determined by the EMID's score matrix rather than majority human vote, which may bias results if the dataset's alignment diverges from general human perception.

## Evidence (verbatim from paper)

> Based on the response data of different subjects, we calculate the separate accuracy and overall accuracy of music-to-image and image-to-music experiments. Additionally, we conduct an analysis to investigate the distribution of accuracy across different music emotional categories. Note that accuracy refers to the ratio of the number of consistent situations between the subjects' answers and pairs in the EMID to the total number of validation questions.

## Citation

```bibtex
@misc{zou2023emid,
  title={EMID: An Emotional Aligned Dataset in Audio-Visual Modality},
  author={Zou et al. (2023)},
  year={2023},
  note={arXiv:2308.07622}
}
```

- arXiv: 2308.07622