Emotiw2018 Group Er Eval

---
name: emotiw2018-group-er-eval
description: Evaluates a model's ability to recognize and aggregate facial emotions across multiple individuals in a crowd scene. It probes the system's capacity to capture complex spatial dependencies among overlapping facial expressions to predict the dominant group-level emotion. Use when the user wants to benchmark on EmotiW2018, GECV, or asks about evaluating this task. Reports mean accuracy (mAC).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1811.11849
  bibtex_key: quach2018nvpf
  confidence: high
---

# emotiw2018-group-er-eval

> Non-Volume Preserving-based Fusion to Group-Level Emotion Recognition on Crowd Videos — Quach et al. (2018) (arXiv:1811.11849, 2018)

## What this evaluates

Evaluates a model's ability to recognize and aggregate facial emotions across multiple individuals in a crowd scene. It probes the system's capacity to capture complex spatial dependencies among overlapping facial expressions to predict the dominant group-level emotion.

## Datasets

- **EmotiW2018** — total 17172; splits: train (9815), val (4346), test (3011)
- **GECV** — total 627; splits: train (565), test (62)

## Metrics

- `mean accuracy (mAC)` **(primary)** — range: [0, 1]
  - Average of per-class accuracies across Neutral, Positive, and Negative emotion categories.
- `Unweighted Average Recall (UAR)` — range: [0, 1]
  - Average of recall scores computed independently for each emotion class, treating all classes equally regardless of sample size.
- `F1-score` — range: [0, 1]
  - Harmonic mean of precision and recall, typically reported as the macro-average across classes.

## Input / output format

**Input**: Crowd images or video frames containing multiple faces, with ground-truth group-level emotion labels.

**Output**: A single predicted group-level emotion label per image/video (Neutral, Positive, or Negative).

## Scoring recipe

```python
def compute_metrics(predictions, gold):
    classes = ['Neutral', 'Positive', 'Negative']
    class_accuracies = []
    class_recalls = []
    for cls in classes:
        tp = sum(1 for p, g in zip(predictions, gold) if p == cls and g == cls)
        fn = sum(1 for p, g in zip(predictions, gold) if p != cls and g == cls)
        class_accuracies.append(tp / (tp + fn) if (tp + fn) > 0 else 0)
        class_recalls.append(tp / (tp + fn) if (tp + fn) > 0 else 0)
    mAC = sum(class_accuracies) / len(classes)
    UAR = sum(class_recalls) / len(classes)
    return mAC, UAR
```

## Common pitfalls

- Fusion strategy choice (averaging vs. concatenation vs. NVPF) drastically changes performance, so baselines must use identical feature extractors.
- Class imbalance in crowd scenes can skew overall accuracy; UAR or F1 should be checked to ensure minority emotions (e.g., Negative) are not ignored.
- Low-resolution and overlapping faces in crowd videos require robust feature extraction before fusion.

## Evidence (verbatim from paper)

> Overall/mean accuracy, per class accuracy, mean F1, and Unweighted Average Recall (UAR) are reported in this experiment. EmotiW 2018 group-based ER sub-challenge is an extension of the sub-challenge in EmotiW 2017 with 9,815 images for training, 4,346 images for validation, and 3,011 for testing, respectively.

## Citation

```bibtex
@misc{quach2018nvpf,
  title={Non-Volume Preserving-based Fusion to Group-Level Emotion Recognition on Crowd Videos},
  author={Quach et al. (2018)},
  year={2018},
  note={arXiv:1811.11849}
}
```

- arXiv: 1811.11849