Lrl Spoof Srr Eval

---
name: lrl-spoof-srr-eval
description: Evaluates cross-lingual robustness of spoofing countermeasures by measuring spoof rejection rates on a multilingual synthetic-speech corpus at a fixed operating point calibrated on external benchmarks. Probes how language and synthesizer identity independently affect spoof detection performance. Use when the user wants to benchmark on Low-Resource Language Spoofing Corpus, or asks about evaluating this task. Reports spoof rejection rate (SRR).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.02364
  bibtex_key: borodin2026whenspoof
  confidence: high
---

# lrl-spoof-srr-eval

> When Spoof Detectors Travel: Evaluation Across 66 Languages in the Low-Resource Language Spoofing Corpus — Borodin et al. (2026) (arXiv:2603.02364, 2026)

## What this evaluates

Evaluates cross-lingual robustness of spoofing countermeasures by measuring spoof rejection rates on a multilingual synthetic-speech corpus at a fixed operating point calibrated on external benchmarks. Probes how language and synthesizer identity independently affect spoof detection performance.

## Datasets

- **Low-Resource Language Spoofing Corpus** — total ?; splits: test (-1)

## Metrics

- `spoof rejection rate (SRR)` **(primary)** — range: percent
  - SRR(τ) = (1/N) * Σ 1[s(x_i) < τ], where s(x) is the CM score (higher indicates bonafide), τ is a fixed threshold, and x_i are spoofed utterances. Evaluated at a transferred EER operating point τ_EER calibrated on pooled external benchmarks.

## Input / output format

**Input**: Spoofed audio utterances from the target corpus.

**Output**: A continuous CM score per utterance, where higher scores indicate bonafide and lower scores indicate spoofed speech.

## Scoring recipe

```python
# 1. Calibrate threshold on external benchmarks
all_scores = concat_scores(external_benchmarks)
tau_EER = find_threshold_where_FAR_equals_FRR(all_scores)

# 2. Evaluate SRR on target corpus
N = len(target_spoof_utterances)
spoof_scores = [model.predict(x) for x in target_spoof_utterances]
SRR = sum(1 for s in spoof_scores if s < tau_EER) / N
return SRR * 100  # percentage
```

## Common pitfalls

- The target corpus contains only spoofed speech, so standard EER cannot be computed directly on it; a threshold must be transferred from external benchmarks.
- SRR is a spoof-only metric and does not account for false rejection rates on bonafide speech, so it does not represent a full security operating point.
- Performance differences across languages may be confounded by synthesizer identity unless CM and TTS systems are explicitly fixed during controlled comparisons.

## Evidence (verbatim from paper)

> Let $s(x)$ denote the CM score for utterance $x$; higher scores indicate bonafide. Given a threshold $\tau$, the spoof rejection rate (SRR) on our spoof-only corpus is $\mathrm{SRR}(\tau)\=\frac{1}{N}\sum_{i\=1}^{N}\mathbbm{1}\big[s(x_{i})<\tau\big],$ where ${x_{i}}_{i\=1}^{N}$ are spoofed utterances. Since our corpus contains no bonafide speech, we cannot compute an EER directly on it; instead, we evaluate SRR at a transferred operating point $\tau_{\mathrm{EER}}$ calibrated on pooled external benchmarks (Section 4.3).

## Citation

```bibtex
@misc{borodin2026whenspoof,
  title={When Spoof Detectors Travel: Evaluation Across 66 Languages in the Low-Resource Language Spoofing Corpus},
  author={Borodin et al. (2026)},
  year={2026},
  note={arXiv:2603.02364}
}
```

- arXiv: 2603.02364