Ucla Asv Eval

---
name: ucla-asv-eval
description: Evaluates automatic speaker verification (ASV) robustness to speaking-style mismatches between enrollment and test utterances. It measures how well data augmentation techniques can compensate for style variability without requiring multi-style training data. Use when the user wants to benchmark on UCLA database, or asks about evaluating this task. Reports EER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2008.03616
  bibtex_key: afshan2020variable
  confidence: high
---

# ucla-asv-eval

> Variable frame rate-based data augmentation to handle speaking-style variability for automatic speaker verification — Afshan et al. (2020) (arXiv:2008.03616, 2020)

## What this evaluates

Evaluates automatic speaker verification (ASV) robustness to speaking-style mismatches between enrollment and test utterances. It measures how well data augmentation techniques can compensate for style variability without requiring multi-style training data.

## Datasets

- **UCLA database** — total 240; splits: development (435), enrollment (252), test (240)

## Metrics

- `EER` **(primary)** — range: percent
  - Equal Error Rate: the operating point where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). Computed across a range of decision thresholds and reported as a percentage.

## Input / output format

**Input**: 30-second speech utterances for enrollment and test pairs. Features (e.g., x-vectors) are extracted and processed by a PLDA classifier.

**Output**: A verification score or binary decision per enrollment-test pair, aggregated to compute FAR/FRR and EER across thresholds.

## Scoring recipe

```python
def compute_eer(scores, labels, thresholds):
    far, frr = [], []
    for t in thresholds:
        far.append(mean((scores[labels==1] < t)))
        frr.append(mean((scores[labels==0] > t)))
    idx = argmin(abs(array(far) - array(frr)))
    return far[idx] * 100
```

## Common pitfalls

- Style-mismatched EERs are significantly higher than style-matched ones; readers must carefully match enrollment/test pairs in Table 2 to avoid misinterpreting baseline performance.
- VFR normalization is only applied to the development set for PLDA adaptation, not to enrollment/test utterances, to preserve speaker-specific information, which limits its direct comparability to full normalization methods.
- The multi-style configuration serves as a best-case upper bound but is noted as unrealistic for practical deployment where all speaking styles per speaker are unavailable.

## Evidence (verbatim from paper)

> System performance in terms of the EER for the UCLA database is shown in Table 2. Statistical significance was verified using McNemar’s test. Unless mentioned explicitly, all performance differences reported in this section are significant with p<0.005.

## Citation

```bibtex
@misc{afshan2020variable,
  title={Variable frame rate-based data augmentation to handle speaking-style variability for automatic speaker verification},
  author={Afshan et al. (2020)},
  year={2020},
  note={arXiv:2008.03616}
}
```

- arXiv: 2008.03616