T2i Deanonymization Eval

---
name: t2i-deanonymization-eval
description: Evaluates the ability to deanonymize text-to-image models by identifying which model generated a given image, exploiting model-specific visual signatures in embedding space. Use when the user wants to benchmark on T2I Leaderboard Prompts, or asks about evaluating this task. Reports Top-1 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.09647
  bibtex_key: naseh2026identifying
  confidence: high
---

# t2i-deanonymization-eval

> Identifying Models Behind Text-to-Image Leaderboards — Naseh et al. (2026) (arXiv:2601.09647, 2026)

## What this evaluates

Evaluates the ability to deanonymize text-to-image models by identifying which model generated a given image, exploiting model-specific visual signatures in embedding space.

## Datasets

- **T2I Leaderboard Prompts** — total 280; splits: test (280)

## Metrics

- `Top-1 accuracy` **(primary)** — range: [0, 1]
  - Fraction of test images for which the correct generating model appears within the top-k nearest centroids in the image embedding space.

## Input / output format

**Input**: Generated images from text-to-image models, optionally paired with the text prompt used to generate them.

**Output**: Predicted model identifier (for k-NN classification) or binary decision (for one-vs-rest thresholding).

## Scoring recipe

```python
def topk_accuracy(predictions, gold, k=1):
    correct = sum(1 for p, g in zip(predictions, gold) if g in p[:k])
    return correct / len(gold)
```

## Common pitfalls

- The main k-NN evaluation assumes the adversary can query all model APIs to build centroids, though a one-vs-rest variant relaxes this.
- Results are averaged over 5 randomized prompt-model assignments to account for generation variability, which must be replicated for fair comparison.
- Performance is highly sensitive to the choice of image encoder (e.g., CLIP vs. SigLIP), as noted in the ablation study.

## Evidence (verbatim from paper)

> We evaluate all attacks using top-1, top-2, and top-3 identification accuracy. These metrics quantify the fraction of test images for which the correct generating model appears within the top-k nearest centroids.

## Citation

```bibtex
@misc{naseh2026identifying,
  title={Identifying Models Behind Text-to-Image Leaderboards},
  author={Naseh et al. (2026)},
  year={2026},
  note={arXiv:2601.09647}
}
```

- arXiv: 2601.09647