Ser Multiwindow Eval

---
name: ser-multiwindow-eval
description: Evaluates deep learning models for speech emotion recognition (SER) using a multi-window data augmentation strategy. It probes the model's ability to classify categorical emotions from speech audio under varying feature extraction window sizes and class configurations. Use when the user wants to benchmark on IEMOCAP, RAVDESS, SAVEE, or asks about evaluating this task. Reports Unweighted Accuracy (UA).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2010.09895
  bibtex_key: padi2020multiwindow
  confidence: high
---

# ser-multiwindow-eval

> Multi-Window Data Augmentation Approach for Speech Emotion Recognition — Padi et al. (2020) (arXiv:2010.09895, 2020)

## What this evaluates

Evaluates deep learning models for speech emotion recognition (SER) using a multi-window data augmentation strategy. It probes the model's ability to classify categorical emotions from speech audio under varying feature extraction window sizes and class configurations.

## Datasets

- **IEMOCAP** — total 4490; splits: train (-1), test (-1)
- **RAVDESS** — total ?; splits: train (-1), test (-1)
- **SAVEE** — total ?; splits: train (-1), test (-1)

## Metrics

- `Unweighted Accuracy (UA)` **(primary)** — range: percent
  - Macro average of per-class accuracies, calculated as the mean of accuracy scores for each emotion class regardless of class frequency.
- `Weighted Average Precision (WAP)` — range: percent
  - Weighted average of per-class precision scores, where weights correspond to the support (number of true instances) for each class.
- `Weighted Average F1 (WAF1)` — range: percent
  - Weighted average of per-class F1 scores, calculated as the harmonic mean of precision and recall for each class, averaged using class support as weights.

## Input / output format

**Input**: Raw speech audio files, processed into speech-based features using multiple window sizes (25ms, 50ms, 100ms, 200ms) with 50% overlap.

**Output**: Categorical emotion label (e.g., 'angry', 'happy', 'neutral', 'sad', 'excited', 'fear', 'disgust').

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    classes = sorted(set(y_true + y_pred))
    class_accs = [sum(t==p for t,p in zip(y_true, y_pred) if t==c) / max(sum(1 for t in y_true if t==c), 1) for c in classes]
    ua = sum(class_accs) / len(classes)
    precisions = [precision_score(y_true, y_pred, labels=[c], average=None)[0] for c in classes]
    recalls = [recall_score(y_true, y_pred, labels=[c], average=None)[0] for c in classes]
    f1s = [2 * p * r / (p + r) if (p + r) > 0 else 0 for p, r in zip(precisions, recalls)]
    supports = [sum(1 for t in y_true if t == c) for c in classes]
    total_support = sum(supports)
    wap = sum(p * s for p, s in zip(precisions, supports)) / total_support
    waf1 = sum(f * s for f, s in zip(f1s, supports)) / total_support
    return ua, wap, waf1
```

## Common pitfalls

- Do not use cross-validation; the paper explicitly uses a fixed 80/20 train/test split due to computational constraints.
- For RAVDESS, only spoken sentences should be used; sung sentences must be excluded.
- RAVDESS requires merging 'neutral' and 'calm' into a single 'neutral' class.
- IEMOCAP has two distinct class configurations (Exp 1: happy; Exp 2: excited) that must not be mixed.

## Evidence (verbatim from paper)

> In our evaluations, we use 80% of the data for training and 20% of data for testing purposes because performing cross-validation on deep learning models with varying window sizes is not feasible in terms of time and computational requirements. We present our findings by reporting Unweighted Accuracy (UA), Weighted Average Precision (WAP), and Weighted Average F1 (WAF1) measures.

## Citation

```bibtex
@misc{padi2020multiwindow,
  title={Multi-Window Data Augmentation Approach for Speech Emotion Recognition},
  author={Padi et al. (2020)},
  year={2020},
  note={arXiv:2010.09895}
}
```

- arXiv: 2010.09895