Libritts Selfvc Watermark Eval

---
name: libritts-selfvc-watermark-eval
description: Evaluates the robustness of neural audio watermarking systems against self voice conversion attacks and transmission channel distortions, while measuring speaker identity preservation, linguistic content integrity, and perceptual quality. Use when the user wants to benchmark on LibriTTS, or asks about evaluating this task. Reports bitwise extraction accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.20432
  bibtex_key: [REDACTED]
  confidence: high
---

# libritts-selfvc-watermark-eval

> Self Voice Conversion as an Attack against Neural Audio Watermarking — Özer et al. (2026) (arXiv:2601.20432, 2026)

## What this evaluates

Evaluates the robustness of neural audio watermarking systems against self voice conversion attacks and transmission channel distortions, while measuring speaker identity preservation, linguistic content integrity, and perceptual quality.

## Datasets

- **LibriTTS** — total ?; splits: test-clean (-1)

## Metrics

- `bitwise extraction accuracy` **(primary)** — range: [0, 1]
  - Reported as 1 - bitwise extraction accuracy. Values near 0 indicate perfect extraction; values near 0.5 indicate random guessing (watermark failure).
- `WER` — range: percent
  - Word Error Rate computed by Whisper-L ASR against ground-truth text.
- `UTMOS` — range: other
  - Neural mean opinion score predictor for perceptual speech quality.
- `speaker similarity` — range: other
  - Cosine similarity between ECAPA speaker embeddings extracted from reference and converted utterances.

## Input / output format

**Input**: Audio utterances from LibriTTS test-clean, optionally processed through self-VC models (kNN-VC, RVC), baseline vocoders, or random compound transmission distortions (background noise, Gaussian noise, resampling, lossy compression).

**Output**: Metric scores per utterance or averaged across the split: bitwise extraction accuracy, WER, UTMOS, and speaker similarity.

## Scoring recipe

```python
# Speaker Similarity
sim = cosine_similarity(ecapa_embed(ref_audio), ecapa_embed(vc_audio))
# Content Preservation
wer = whisper_l_wer(vc_audio, gt_text)
# Perceptual Quality
utmos = utmos_predictor(vc_audio)
# Watermark Robustness
extracted_bits = watermark_decoder(vc_audio)
bitwise_error = 1.0 - bitwise_accuracy(extracted_bits, original_bits)
```

## Common pitfalls

- Confusing transmission channel distortions (noise, compression) with the proposed adversarial self-VC attacks, which are content-preserving.
- Misinterpreting the error scale: values near 0.5 indicate watermark failure (random guessing), not success.
- Assuming high perceptual quality (UTMOS) or low WER implies the watermark survives; self-VC breaks watermarks while maintaining intelligibility and naturalness.

## Evidence (verbatim from paper)

> Content preservation is evaluated using the Whisper-L automatic speech recognition (ASR) model, where word error rates (WER) are computed against the ground-truth (GT) text. Finally, we assess perceptual speech quality using UTMOS, a neural mean opinion score (MOS) predictor commonly used in TTS and VC evaluation.

## Citation

```bibtex
@misc{ozer2026selfvoice,
  title={Self Voice Conversion as an Attack against Neural Audio Watermarking},
  author={Özer et al. (2026)},
  year={2026},
  note={arXiv:2601.20432}
}
```

- arXiv: 2601.20432