Hula Antispoofing Eval

---
name: hula-antispoofing-eval
description: This benchmark evaluates anti-spoofing systems for synthetic speech detection, with a specific focus on prosody-awareness, emotional/expressive spoofing, and cross-lingual robustness. It tests whether models can distinguish real speech from TTS, VC, and adversarial attacks across diverse channel conditions and languages. Use when the user wants to benchmark on ASVspoof 2019 (LA), ASVspoof 2021 (LA), ASVspoof 2024 (Track 1), EmoFake, Mixed Emotions, ADD 2022 (Track 1), HABLA, or asks about evaluating this task. Reports EER%.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.21676
  bibtex_key: [REDACTED]
  confidence: high
---

# hula-antispoofing-eval

> HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech — Mahapatra et al. (2025) (arXiv:2509.21676, 2025)

## What this evaluates

This benchmark evaluates anti-spoofing systems for synthetic speech detection, with a specific focus on prosody-awareness, emotional/expressive spoofing, and cross-lingual robustness. It tests whether models can distinguish real speech from TTS, VC, and adversarial attacks across diverse channel conditions and languages.

## Datasets

- **ASVspoof 2019 (LA)** — total 71237; splits: test (71237)
- **ASVspoof 2021 (LA)** — total 148176; splits: test (148176)
- **ASVspoof 2024 (Track 1)** — total 680774; splits: test (680774)
- **EmoFake** — total 17500; splits: test (17500)
- **Mixed Emotions** — total 491; splits: test (491)
- **ADD 2022 (Track 1)** — total 109199; splits: test (109199)
- **HABLA** — total 75816; splits: test (75816)

## Metrics

- `EER%` **(primary)** — range: percent
  - Equal Error Rate (EER) is the operating point where the False Acceptance Rate (FAR) and False Rejection Rate (FRR) are equal. It is reported as a percentage.

## Input / output format

**Input**: Audio clips trimmed or zero-padded to approximately 4 seconds. Processed through an XLS-R 300M backbone with 25 ms frame length and 20 ms frame period.

**Output**: Binary spoof detection score (probability of being synthetic), thresholded to compute EER%.

## Scoring recipe

```python
def compute_eer(scores, labels):
    # scores: float array (higher = more likely spoof)
    # labels: 1 for spoof, 0 for bona fide
    far, frr = compute_roc_curve(scores, labels)
    idx = np.argmin(np.abs(far - frr))
    return far[idx] * 100  # EER in percent
```

## Common pitfalls

- EER% is highly sensitive to the decision threshold; systems must report the exact operating point or use min-CER/HTTER conventions for fair comparison.
- Cross-dataset evaluation (e.g., training on ASVspoof 2019 and testing on 2021/2024 or EmoFake) often reveals severe domain shift; models may overfit to specific TTS/VC systems used in training.
- Prosodic features (F0, voiced/unvoiced) are critical for emotional spoofing detection; ignoring them leads to high error rates on EmoFake and Mixed Emotions.

## Evidence (verbatim from paper)

> We evaluate HuLA across a diverse set of corpora, using two datasets for training and the rest only for evaluation. This design ensures that performance is assessed under realistic, out-of-domain conditions. TABLE IV: Comparison of baseline and proposed models on the ASVspoof evaluation sets in terms of EER%.

## Citation

```bibtex
@misc{mahapatra2025hula,
  title={HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech},
  author={Mahapatra et al. (2025)},
  year={2025},
  note={arXiv:2509.21676}
}
```

- arXiv: 2509.21676