L2 Arctic Accent Eval

---
name: l2-arctic-accent-eval
description: Evaluates a streaming foreign accent conversion system's ability to neutralize non-native pronunciation while preserving speaker identity. The protocol uses self-reconstruction mode and compares synthesized outputs against offline-generated golden speaker utterances as a reference baseline. Use when the user wants to benchmark on L2-ARCTIC (Indian subset), or asks about evaluating this task. Reports non-native accent confidence.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.27001
  bibtex_key: quamer2026phonos
  confidence: medium
---

# l2-arctic-accent-eval

> PHONOS: PHOnetic Neutralization for Online Streaming Applications — Quamer et al. (2026) (arXiv:2603.27001, 2026)

## What this evaluates

Evaluates a streaming foreign accent conversion system's ability to neutralize non-native pronunciation while preserving speaker identity. The protocol uses self-reconstruction mode and compares synthesized outputs against offline-generated golden speaker utterances as a reference baseline.

## Datasets

- **L2-ARCTIC (Indian subset)** — total 400; splits: test (400)

## Metrics

- `non-native accent confidence` **(primary)** — range: percent
  - Measures the classifier's confidence that the synthesized speech retains non-native accent characteristics. Evaluated as the percentage reduction in confidence scores relative to the original source utterances.

## Input / output format

**Input**: 16 kHz audio utterance with ground truth transcription, conditioned on a speaker embedding extracted from the same source utterance.

**Output**: Synthesized neutralized speech audio.

## Scoring recipe

```python
def score(predictions, gold):
    # predictions: list of synthesized audio files
    # gold: list of corresponding source utterances
    conf_scores = []
    for pred, src in zip(predictions, gold):
        # Run accent classifier on both
        pred_conf = classifier.predict_confidence(pred)
        src_conf = classifier.predict_confidence(src)
        conf_scores.append((src_conf - pred_conf) / src_conf)
    return mean(conf_scores) * 100  # percentage reduction
```

## Common pitfalls

- No open-source baselines available; evaluation relies solely on golden-speaker reference upper bound
- Self-reconstruction mode conditions the model on the exact same speaker embedding as the source utterance
- Golden-speaker utterances are generated offline and discarded after training, serving only as a static reference during evaluation

## Evidence (verbatim from paper)

> achieving an 81% reduction in non-native accent confidence via both classifier and human ratings, with end-to-end GPU latency under 241ms and reduced speaker linkability in embedding space.

## Citation

```bibtex
@misc{quamer2026phonos,
  title={PHONOS: PHOnetic Neutralization for Online Streaming Applications},
  author={Quamer et al. (2026)},
  year={2026},
  note={arXiv:2603.27001}
}
```

- arXiv: 2603.27001