Multiapi Spoof Eval

---
name: multiapi-spoof-eval
description: Evaluates speech anti-spoofing detection and API source attribution capabilities on a large-scale dataset of synthetic speech generated by 30 distinct APIs. It probes model robustness to domain shifts, generalization to unseen spoofing sources, and fine-grained source identification in realistic, heterogeneous environments. Use when the user wants to benchmark on MultiAPI Spoof, or asks about evaluating this task. Reports EER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.07352
  bibtex_key: zhang2025multiapi
  confidence: high
---

# multiapi-spoof-eval

> MultiAPI Spoof: A Multi-API Dataset and Local-Attention Network for Speech Anti-spoofing Detection — Zhang et al. (2025) (arXiv:2512.07352, 2025)

## What this evaluates

Evaluates speech anti-spoofing detection and API source attribution capabilities on a large-scale dataset of synthetic speech generated by 30 distinct APIs. It probes model robustness to domain shifts, generalization to unseen spoofing sources, and fine-grained source identification in realistic, heterogeneous environments.

## Datasets

- **MultiAPI Spoof** — total ?; splits: train (-1), dev (-1), eval (-1); repo https://github.com/XuepingZhang/MultiAPI-Spoof

## Metrics

- `EER` **(primary)** — range: percent
  - Equal Error Rate: the operating threshold at which the false acceptance rate equals the false rejection rate. Lower is better.
- `minDCF` — range: percent
  - Minimum Decision Cost Function: the lowest weighted detection cost across all possible thresholds, calculated using specified prior probabilities and cost parameters. Lower is better.
- `actDCF` — range: percent
  - Actual Decision Cost Function: the weighted detection cost computed at a fixed operating point (typically the EER threshold) using specified priors and costs. Lower is better.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall. For seen APIs, computed as the macro-average across 21 classes. For unseen APIs, computed on the single class. Overall performance is the macro-average across all classes including unseen.

## Input / output format

**Input**: Normalized raw waveforms, uniformly segmented into 4-second clips (shorter clips are repeated, longer clips are truncated). No data augmentation is applied.

**Output**: Anti-spoofing: continuous score or probability indicating spoof likelihood. API Tracing: discrete class label prediction from 22 categories (21 seen APIs + 1 unseen class).

## Scoring recipe

```python
def score_antispoof(preds, labels):
    eer = compute_eer(preds, labels)  # threshold where FAR == FRR
    mindcf = compute_mindcf(preds, labels, prior=0.01, costs=[1, 10])
    actdcf = compute_actdcf(preds, labels, prior=0.01, costs=[1, 10])
    return {'EER': eer, 'minDCF': mindcf, 'actDCF': actdcf}

def score_apitrace(preds, labels):
    prec, rec, f1 = precision_recall_f1_score(labels, preds, average='macro')
    return {'precision': prec, 'recall': rec, 'F1': f1}
```

## Common pitfalls

- Models trained exclusively on traditional public datasets (e.g., TIMIT, ASVspoof) exhibit severe domain shift and high EER on MultiAPI Spoof without additional domain-specific training data.
- API tracing suffers from low recall on unseen APIs despite high precision, as embeddings of unseen APIs mix with seen categories rather than forming separable clusters.

## Evidence (verbatim from paper)

> For the anti-spoofing task, performance is evaluated using Equal Error Rate (EER↓), minimum Decision Cost Function (minDCF↓), and actual Decision Cost Function (actDCF↓) [38]. For the API tracing task, we measure classification performance using precision, recall, and F1 [39]. Specifically, the F1 for seen APIs is computed as the macro-average of the F1 scores over the 21 seen APIs classes. For unseen APIs, F1 is computed on the single unseen class. The overall performance is reported as the macro-average across all classes, including the unseen class.

## Citation

```bibtex
@misc{zhang2025multiapi,
  title={MultiAPI Spoof: A Multi-API Dataset and Local-Attention Network for Speech Anti-spoofing Detection},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2512.07352}
}
```

- arXiv: 2512.07352