Blmore Challenge Eval

---
name: blmore-challenge-eval
description: Evaluates multimodal emotion recognition systems on detecting blended emotions (presence and salience) across unseen actors. It probes the model's ability to generalize actor-invariant emotional semantics from audio, visual, and combined modalities under a strict threshold-based discretization protocol. Use when the user wants to benchmark on BLEMORE, or asks about evaluating this task. Reports Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.23650
  bibtex_key: chapariniya2026blended
  confidence: high
---

# blmore-challenge-eval

> Foundation Model Embeddings Meet Blended Emotions: A Multimodal Fusion Approach for the BLEMORE Challenge — Chapariniya et al. (2026) (arXiv:2603.23650, 2026)

## What this evaluates

Evaluates multimodal emotion recognition systems on detecting blended emotions (presence and salience) across unseen actors. It probes the model's ability to generalize actor-invariant emotional semantics from audio, visual, and combined modalities under a strict threshold-based discretization protocol.

## Datasets

- **BLEMORE** — total 3050; splits: train (2456), test (594)

## Metrics

- `ACC_P` — range: [0, 1]
  - Presence accuracy: fraction of clips where the discretized presence prediction (score ≥ α) matches the ground truth presence label.
- `ACC_S` — range: [0, 1]
  - Salience accuracy: fraction of clips where the discretized salience prediction (score ≥ β) matches the ground truth salience label.
- `Score` **(primary)** — range: [0, 1]
  - Official BLEMORE challenge metric. Continuous outputs are thresholded using α (presence) and β (salience) via grid search on validation data. Score is the official challenge metric (typically the average of ACC_P and ACC_S).

## Input / output format

**Input**: Video clips containing audio and visual modalities (face, body, or full-body sequences).

**Output**: Continuous presence and salience scores per emotion class, discretized via thresholds α (presence) and β (salience).

## Scoring recipe

```python
def compute_blemore_metrics(predictions, ground_truth, alpha, beta):
    pred_presence = (predictions.presence >= alpha).astype(int)
    pred_salience = (predictions.salience >= beta).astype(int)
    acc_p = (pred_presence == ground_truth.presence).mean()
    acc_s = (pred_salience == ground_truth.salience).mean()
    score = (acc_p + acc_s) / 2
    return acc_p, acc_s, score
```

## Common pitfalls

- Threshold β instability: Optimal salience threshold varies drastically (0.05–0.43) across folds due to actor-specific expression diversity, causing significant validation-to-test performance gaps.
- Actor-disjoint evaluation: Test set uses completely unseen actors, making models prone to overfitting to actor-specific visual/audio cues rather than general emotional semantics.

## Evidence (verbatim from paper)

> Following the official BLEMORE protocol, continuous outputs are discretized via presence threshold α and salience threshold β, selected by grid search on validation data. ... Our 12-encoder ensemble with Gemini achieves ACC_P = 0.391, substantially outperforming the best baseline presence (VideoMAEv2 + HuBERT, 0.332).

## Citation

```bibtex
@misc{chapariniya2026blended,
  title={Foundation Model Embeddings Meet Blended Emotions: A Multimodal Fusion Approach for the BLEMORE Challenge},
  author={Chapariniya et al. (2026)},
  year={2026},
  note={arXiv:2603.23650}
}
```

- arXiv: 2603.23650