Text Image Retrieval Eval

---
name: text-image-retrieval-eval
description: Evaluates the model's ability to align facial images with their textual descriptions by retrieving the correct image given a text query, and vice versa. It measures how well the model learns cross-modal semantic correspondence for face-centric data. Use when the user wants to benchmark on CelebA-Caption, MM-CelebA, or asks about evaluating this task. Reports R@5, R@10.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.08515
  bibtex_key: dai2024facecaption
  confidence: high
---

# text-image-retrieval-eval

> 15M Multimodal Facial Image-Text Dataset — Dawei Dai et al. (2024) (arXiv:2407.08515, 2024)

## What this evaluates

Evaluates the model's ability to align facial images with their textual descriptions by retrieving the correct image given a text query, and vice versa. It measures how well the model learns cross-modal semantic correspondence for face-centric data.

## Datasets

- **CelebA-Caption** — total 182732; splits: train (162770), test (19962)
- **MM-CelebA** — total 30000; splits: train (24000), test (6000)

## Metrics

- `R@5, R@10` **(primary)** — range: percent
  - Recall at top-K, measuring the percentage of queries where the ground-truth image (or text) appears in the top-K retrieved results. Evaluated in both Text->Image and Image->Text directions.

## Input / output format

**Input**: A facial image and its corresponding caption text (or vice versa) used as a query against a candidate pool.

**Output**: A ranked list of retrieved images or texts from the candidate pool.

## Scoring recipe

```python
def compute_rk(predictions, gold, k):
    ranks = []
    for pred_list, gold_item in zip(predictions, gold):
        if gold_item in pred_list:
            ranks.append(pred_list.index(gold_item) + 1)
        else:
            ranks.append(k + 1)
    return sum(1 for r in ranks if r <= k) / len(ranks) * 100
```

## Common pitfalls

- Freezing backbone weights and only fine-tuning linear layers may limit performance compared to full fine-tuning baselines.
- Retrieval is evaluated in both directions (Text->Image and Image->Text), which must be reported separately and are not interchangeable.

## Evidence (verbatim from paper)

> We adopted the Top-K retrieval index, where R@5 and R@10 represent the top-5 and top-10 accuracy, respectively.

## Citation

```bibtex
@misc{dai2024facecaption,
  title={15M Multimodal Facial Image-Text Dataset},
  author={Dawei Dai et al. (2024)},
  year={2024},
  note={arXiv:2407.08515}
}
```

- arXiv: 2407.08515