Token Embedding Inversion Accuracy

---
name: token-embedding-inversion-accuracy
description: Evaluates the privacy leakage of a token-level perturbation mechanism by measuring how easily an adversary can recover original tokens from their privatized embeddings. It probes the robustness of the privacy-preserving noise injection against nearest-neighbor-based inversion attacks. Use when the user has predictions and gold and needs to compute token embedding inversion accuracy.
metadata:
  skill_kind: metric
  source_arxiv: 2104.07504
  bibtex_key: chen2021privacybert
  confidence: high
---

# token-embedding-inversion-accuracy

> Natural Language Understanding with Privacy-Preserving BERT — Chen Qu et al. (2021) (arXiv:2104.07504, 2021)

## What this evaluates

Evaluates the privacy leakage of a token-level perturbation mechanism by measuring how easily an adversary can recover original tokens from their privatized embeddings. It probes the robustness of the privacy-preserving noise injection against nearest-neighbor-based inversion attacks.

## Datasets

- **SST** — total ?; splits: validation (-1)
- **QQP** — total ?; splits: validation (-1)

## Metrics

- `token embedding inversion accuracy` **(primary)** — range: [0, 1]
  - Fraction of perturbed tokens correctly recovered by finding the nearest neighbor in the BERT embedding space.
- `N_w` — range: [0, 1]
  - Probability of an input token not being modified by the perturbation mechanism, estimated as the fraction of 1,000 simulations where the output token matches the input.
- `S_w` — range: other
  - Effective support of the output distribution, estimated as the number of unique output tokens across 1,000 simulations.

## Input / output format

**Input**: Perturbed token embeddings generated by the privacy mechanism

**Output**: Predicted original token

## Scoring recipe

```python
def compute_inversion_accuracy(perturbed_embs, original_tokens, vocab_embs):
    correct = 0
    for pert_emb, orig_tok in zip(perturbed_embs, original_tokens):
        pred_tok = argmin([cosine_dist(pert_emb, v_emb) for v_emb in vocab_embs])
        if pred_tok == orig_tok:
            correct += 1
    return correct / len(original_tokens)

def compute_plausible_deniability(token, vocab, n_sim=1000):
    outputs = [perturb(token) for _ in range(n_sim)]
    N_w = sum(1 for o in outputs if o == token) / n_sim
    S_w = len(set(outputs))
    return N_w, S_w
```

## Common pitfalls

- Measures privacy leakage, not downstream NLU task utility or model accuracy.
- Relies solely on nearest neighbor search; does not evaluate more advanced ML inversion attacks.
- Plausible deniability statistics (N_w, S_w) are estimated via Monte Carlo simulation (1,000 perturbations per token) rather than empirical dataset frequencies.

## Evidence (verbatim from paper)

> We define an adversarial task, token embedding inversion, as recovering the original tokens based on perturbed token embeddings. We leverage nearest neighbor search for this task. Given a perturbed token embedding, we find the nearest neighbor of this embedding in the embedding space as the predicted original token. The performance is measured by accuracy.

## Citation

```bibtex
@misc{chen2021privacybert,
  title={Natural Language Understanding with Privacy-Preserving BERT},
  author={Chen Qu et al. (2021)},
  year={2021},
  note={arXiv:2104.07504}
}
```

- arXiv: 2104.07504