Thai Ser Eval

---
name: thai-ser-eval
description: Evaluates speech emotion recognition models on a culturally grounded Thai speech corpus, testing their ability to classify utterances into five emotion categories (neutral, angry, happy, sad, frustrated) across different recording environments and cross-corpus settings. Use when the user wants to benchmark on THAI-SER, or asks about evaluating this task. Reports weighted accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.09618
  bibtex_key: [REDACTED]
  confidence: high
---

# thai-ser-eval

> THAI Speech Emotion Recognition (THAI-SER) corpus — Wongpithayadisai et al. (2025) (arXiv:2507.09618, 2025)

## What this evaluates

Evaluates speech emotion recognition models on a culturally grounded Thai speech corpus, testing their ability to classify utterances into five emotion categories (neutral, angry, happy, sad, frustrated) across different recording environments and cross-corpus settings.

## Datasets

- **THAI-SER** — total 27854; splits: train (-1), val (-1), test (-1); repo https://github.com/vistec-AI/dataset-releases

## Metrics

- `weighted accuracy` **(primary)** — range: percent
  - Computed by dividing the number of correctly classified samples by the total number of samples.
- `unweighted accuracy` — range: percent
  - Calculated by taking the average of per-class accuracy across all emotion categories.

## Input / output format

**Input**: 64 mel-scale filterbank coefficients (25ms frame length, 10ms shift), split into 3-second intervals (padded if shorter), normalized via CMVN, with VTLP augmentation applied to training samples.

**Output**: Discrete emotion class label from {neutral, angry, happy, sad, frustrated} (or a 4-class subset excluding frustrated).

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels):
    total = len(gold_labels)
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    weighted_acc = correct / total
    
    classes = set(gold_labels)
    per_class_acc = []
    for cls in classes:
        cls_mask = [g == cls for g in gold_labels]
        if sum(cls_mask) > 0:
            cls_correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g and g == cls)
            per_class_acc.append(cls_correct / sum(cls_mask))
    unweighted_acc = sum(per_class_acc) / len(per_class_acc)
    
    return weighted_acc, unweighted_acc
```

## Common pitfalls

- Zoom recordings are explicitly excluded from the baseline k-fold cross-validation due to domain mismatch and audio compression artifacts; they are reserved as a separate out-of-domain challenge set.
- Training data must be filtered using an agreement score threshold of 0.71 to remove samples with low inter-annotator reliability, which otherwise confuses the model.
- Cross-corpus comparisons (e.g., with IEMOCAP) require pruning both datasets to match speaker counts and total hours per fold to avoid unfair size imbalances.

## Evidence (verbatim from paper)

> We chose weighted accuracy and unweighted accuracy as the evaluation metrics. These two metrics have been used in various literature (Satt, Rozenberg, and Hoory (2017), Ando et al. (2018), and Etienne et al. (2018)). The weighted accuracy is computed by dividing the number of correctly classified samples by the total number of samples. On the other hand, unweighted accuracy is calculated by taking an average of per-class accuracy.

## Citation

```bibtex
@misc{wongpithayadisai2025thai,
  title={THAI Speech Emotion Recognition (THAI-SER) corpus},
  author={Wongpithayadisai et al. (2025)},
  year={2025},
  note={arXiv:2507.09618}
}
```

- arXiv: 2507.09618