Sea Spoof Eval

---
name: sea-spoof-eval
description: This benchmark evaluates audio deepfake detection models on their ability to distinguish real speech from synthetic speech across six South-East Asian languages. It specifically probes cross-lingual generalization and robustness against diverse open-source and commercial text-to-speech and voice conversion systems. Use when the user wants to benchmark on SEA-Spoof, or asks about evaluating this task. Reports EER (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.19865
  bibtex_key: wu2025seaspoof
  confidence: high
---

# sea-spoof-eval

> SEA-Spoof: Bridging The Gap in Multilingual Audio Deepfake Detection for South-East Asian — Jinyang Wu et al. (2025) (arXiv:2509.19865, 2025)

## What this evaluates

This benchmark evaluates audio deepfake detection models on their ability to distinguish real speech from synthetic speech across six South-East Asian languages. It specifically probes cross-lingual generalization and robustness against diverse open-source and commercial text-to-speech and voice conversion systems.

## Datasets

- **SEA-Spoof** — total 711; splits: train (439000), val (57000), test (57000)

## Metrics

- `EER (%)` **(primary)** — range: percent
  - Equal Error Rate is the operating point where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). It is computed by sweeping a decision threshold over the model's output scores and finding the intersection of the FAR and FRR curves, then expressed as a percentage.

## Input / output format

**Input**: Raw audio clips (bona-fide or spoofed) paired with binary authenticity labels.

**Output**: Binary classification prediction (real vs. spoof) or a continuous spoof probability score used to compute detection thresholds.

## Scoring recipe

```python
def compute_eer(scores, labels):
    # scores: model spoof probability, labels: 1=spoof, 0=bona-fide
    far, frr, _ = compute_roc_curve(scores, labels)
    idx = np.argmin(np.abs(far - frr))
    return far[idx] * 100
```

## Common pitfalls

- Models trained on Western benchmarks (e.g., ASVspoof) suffer severe cross-lingual performance collapse when evaluated on SEA languages without fine-tuning.
- Fine-tuning on SEA-Spoof can cause catastrophic forgetting, degrading performance on original training benchmarks like ASVspoof5.
- Commercial TTS/VC systems generate significantly harder fakes than open-source models, leading to higher EERs if not accounted for in split design.

## Evidence (verbatim from paper)

> Results (Table[2]) reveal severe cross-domain mismatch: MoLEx achieves strong performance on ASVspoof5 (1.25% EER) but collapses on SEA-Spoof (43.8% EER).

## Citation

```bibtex
@misc{wu2025seaspoof,
  title={SEA-Spoof: Bridging The Gap in Multilingual Audio Deepfake Detection for South-East Asian},
  author={Jinyang Wu et al. (2025)},
  year={2025},
  note={arXiv:2509.19865}
}
```

- arXiv: 2509.19865