Trace Eval

---
name: trace-eval
description: This protocol evaluates training-free partial audio deepfake detection by analyzing the temporal continuity of frozen speech foundation model embeddings. It probes a model's ability to detect splice boundaries and synthetic insertions in speech without requiring labeled training data or architectural modifications. Use when the user wants to benchmark on PartialSpoof, HalfTruth Audio Deepfake (HAD), ADD 2023 Track 2, LlamaPartialSpoof, or asks about evaluating this task. Reports EER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.01083
  bibtex_key: khan2026trace
  confidence: high
---

# trace-eval

> TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models — Awais Khan et al. (arXiv:2604.01083, 2026)

## What this evaluates

This protocol evaluates training-free partial audio deepfake detection by analyzing the temporal continuity of frozen speech foundation model embeddings. It probes a model's ability to detect splice boundaries and synthetic insertions in speech without requiring labeled training data or architectural modifications.

## Datasets

- **PartialSpoof** — total 121461; splits: train (25380), dev (24844), eval (71237)
- **HalfTruth Audio Deepfake (HAD)** — total 79989; splits: train (53093), dev (17824), test (9072)
- **ADD 2023 Track 2** — total 120917; splits: train (53093), dev (17824), test (50000)
- **LlamaPartialSpoof** — total 140616; splits: R01TTS.0.a (76228), R01TTS.0.b (64388)

## Metrics

- `EER` **(primary)** — range: percent
  - Equal Error Rate: the threshold that minimizes the difference between False Acceptance Rate (FAR) and False Rejection Rate (FRR), reported as their average.
- `AUC` — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve, measuring the probability that a randomly chosen spoof utterance scores higher than a bonafide one.
- `Transfer EER` — range: percent
  - EER computed using a fixed threshold calibrated on the PartialSpoof dev set, applied directly to the target test set without target-domain data.
- `Free EER` — range: percent
  - EER computed using the optimal threshold found by searching over the target test set itself.

## Input / output format

**Input**: Raw audio utterances. The system extracts frame-level embeddings at 50 Hz from frozen speech foundation models (e.g., WavLM, HuBERT, Wav2Vec 2.0, Whisper).

**Output**: A continuous anomaly score per utterance, derived from first-order temporal dynamics of the embedding trajectory. A binary decision (bonafide vs. partial fake) is made by comparing the score against a threshold.

## Scoring recipe

```python
def compute_eer(scores, labels):
    # scores: utterance-level anomaly scores
    # labels: 0 (bonafide), 1 (spoof)
    # Find threshold t minimizing |FAR(t) - FRR(t)|
    # FAR = P(score > t | label=0), FRR = P(score <= t | label=1)
    return (FAR(t) + FRR(t)) / 2

def compute_auc(scores, labels):
    return auc(scores, labels)
```

## Common pitfalls

- Confusing 'Transfer EER' (fixed threshold from PartialSpoof dev) with 'Free EER' (threshold optimized on target set).
- HAD test-set bona fide labels are withheld by organizers, so evaluation must strictly use the dev split.
- Fully-fake TTS utterances in LlamaPartialSpoof lack splice boundaries, making them inherently harder for this boundary-focused method and inflating EER (~45%).

## Evidence (verbatim from paper)

> We report Equal Error Rate (EER, %, ↓) and Area Under the ROC Curve (AUC, ↑). For cross-dataset evaluation, we additionally report Transfer EER: the EER obtained by applying the PartialSpoof dev-calibrated threshold directly to the target test set, strictly measuring domain transfer without any target-domain data. Free EER, the EER at the optimal threshold on the target set, serves as an upper bound on in-dataset performance.

## Citation

```bibtex
@misc{khan2026trace,
  title={TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models},
  author={Awais Khan et al.},
  year={2026},
  note={arXiv:2604.01083}
}
```

- arXiv: 2604.01083