Egoscreen Emotion Eval

---
name: egoscreen-emotion-eval
description: Evaluates a model's ability to predict human emotional responses to movie scenes from an egocentric, first-person screen-view perspective. It probes multimodal long-context reasoning by combining visual frames, audio cues, and narrative summaries to handle domain shifts from cinematic to realistic viewing conditions. Use when the user wants to benchmark on EgoScreen-Emotion (ESE), or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.15823
  bibtex_key: dong2026egoscreen
  confidence: high
---

# egoscreen-emotion-eval

> Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions — Ze Dong et al. (2026) (arXiv:2604.15823, 2026)

## What this evaluates

Evaluates a model's ability to predict human emotional responses to movie scenes from an egocentric, first-person screen-view perspective. It probes multimodal long-context reasoning by combining visual frames, audio cues, and narrative summaries to handle domain shifts from cinematic to realistic viewing conditions.

## Datasets

- **EgoScreen-Emotion (ESE)** — total 224; splits: train (-1), test (-1)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Standard classification accuracy: the proportion of correctly predicted emotion labels out of the total number of instances.
- `Macro-F1` — range: percent
  - Macro-averaged F1 score computed across 10 emotion classes, treating each class equally regardless of its frequency in the dataset.
- `Weighted-F1` — range: percent
  - Weighted-averaged F1 score computed across 10 emotion classes, weighting each class's F1 by its support (number of true instances).

## Input / output format

**Input**: Visual input consisting of either 1 or 3 frames sampled at 5-second intervals, optionally augmented with synchronized audio cues and narrative summaries.

**Output**: A single predicted emotion class from a predefined 10-class taxonomy.

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels):
    acc = sum(p == g for p, g in zip(predictions, gold_labels)) / len(gold_labels)
    macro_f1 = macro_f1_score(gold_labels, predictions, average='macro')
    weighted_f1 = weighted_f1_score(gold_labels, predictions, average='weighted')
    return acc, macro_f1, weighted_f1
```

## Common pitfalls

- Models trained on clean cinematic footage suffer a >40% drop in Macro-F1 when evaluated on egocentric FPV data, highlighting a severe domain shift that requires FPV-specific fine-tuning.
- The dataset has a long-tailed label distribution, so Accuracy alone can be misleading; Macro-F1 is reported to fairly evaluate minority emotion classes.
- Input format varies across ablations (1F vs 3F, with/without audio/narrative), so direct comparison requires matching the exact configuration.

## Evidence (verbatim from paper)

> We formulate emotion prediction as a 10-class classification task, deriving ground-truth labels via the confidence-summed aggregation detailed in Sec.[3.2](#S3.SS2 "3.2 Annotation Construction ‣ 3 EgoScreen-Emotion Dataset ‣ Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions"). To rigorously account for the long-tailed distribution, we report Accuracy, Macro-F1, and Weighted-F1.

## Citation

```bibtex
@misc{dong2026egoscreen,
  title={Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companions},
  author={Ze Dong et al. (2026)},
  year={2026},
  note={arXiv:2604.15823}
}
```

- arXiv: 2604.15823