Hsad Eval

---
name: hsad-eval
description: Evaluates audio spoof detection models on a newly constructed hybrid spoofing benchmark. It probes robustness against complex, real-world adversarial conditions including mixed-source speech, environmental noise, channel filtering, and compression artifacts. Use when the user wants to benchmark on Hybrid Spoofed Audio Dataset (HSAD), or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.07323
  bibtex_key: hu2025hsad
  confidence: high
---

# hsad-eval

> When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection — Bin Hu et al. (arXiv:2509.07323, 2025)

## What this evaluates

Evaluates audio spoof detection models on a newly constructed hybrid spoofing benchmark. It probes robustness against complex, real-world adversarial conditions including mixed-source speech, environmental noise, channel filtering, and compression artifacts.

## Datasets

- **Hybrid Spoofed Audio Dataset (HSAD)** — total 42292; splits: test (8500)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Accuracy = (C / N) × 100%, where C is the number of correct predictions and N is the total number of samples.
- `F1-score` — range: [0, 1]
  - Harmonic mean of precision and recall computed across genuine, AI-cloned, AI-generated, and hybrid classes.
- `FPR` — range: [0, 1]
  - False positive rate: proportion of genuine human samples incorrectly classified as spoofed.
- `FNR` — range: [0, 1]
  - False negative rate: proportion of spoofed/hybrid samples incorrectly classified as genuine.

## Input / output format

**Input**: 16 kHz audio recordings converted to 128-bin log-Mel spectrograms (25 ms Hamming window, 10 ms frame shift). Includes clean and degraded (noisy, filtered, compressed) variants.

**Output**: Multi-class label (0: genuine human, 1: AI-cloned, 2: AI-generated, 3: hybrid/mixed) or reliability score.

## Scoring recipe

```python
accuracy = (correct_predictions / total_samples) * 100
fpr = false_positives / (false_positives + true_negatives)
fnr = false_negatives / (false_negatives + true_positives)
f1 = 2 * (precision * recall) / (precision + recall)
reliability_threshold = 0.5
y_hat = 0 if abs(real_tag - reliability_score) < reliability_threshold else 1
```

## Common pitfalls

- Pretrained models frequently collapse to predicting a single dominant class (e.g., AI-generated) without fine-tuning.
- Hybrid/mixed classes cause high prediction variance and overlapping reliability scores, leading to false positives on genuine speech.
- Standard binary spoofing thresholds fail on multi-class hybrid compositions without spoof-specific calibration.

## Evidence (verbatim from paper)

> Performance was measured using classification accuracy, F1-score, false positive rate (FPR), and false negative rate (FNR) across the four spoofing classes. Overall classification accuracy was computed as: Accuracy = (C / N) × 100% where C is the number of correct predictions and N is the total number of samples.

## Citation

```bibtex
@misc{hu2025hsad,
  title={When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection},
  author={Bin Hu et al.},
  year={2025},
  note={arXiv:2509.07323}
}
```

- arXiv: 2509.07323