Multiview Cir Eval

---
name: multiview-cir-eval
description: This benchmark evaluates a model's ability to perform product-level composed image retrieval (CIR) in fashion e-commerce, specifically handling multi-view product images and short modification queries. It probes the model's capacity to align visual perception with textual reasoning across multiple views while filtering out irrelevant gallery items. Use when the user wants to benchmark on DeepFashion, Fashion200K, FashionGen-val, or asks about evaluating this task. Reports Recall@5.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.10297
  bibtex_key: yuan2026fashionmv
  confidence: high
---

# multiview-cir-eval

> FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data — Yuan et al. (2026) (arXiv:2604.10297, 2026)

## What this evaluates

This benchmark evaluates a model's ability to perform product-level composed image retrieval (CIR) in fashion e-commerce, specifically handling multi-view product images and short modification queries. It probes the model's capacity to align visual perception with textual reasoning across multiple views while filtering out irrelevant gallery items.

## Datasets

- **DeepFashion** — total ?; splits: val (-1)
- **Fashion200K** — total ?; splits: val (-1)
- **FashionGen-val** — total ?; splits: val (-1)

## Metrics

- `Recall@5` **(primary)** — range: [0, 1]
  - The proportion of ground-truth target products that appear in the top-5 retrieved results from the gallery.
- `Recall@10` — range: [0, 1]
  - The proportion of ground-truth target products that appear in the top-10 retrieved results from the gallery.

## Input / output format

**Input**: A query consisting of one or more reference product images (multi-view), a short modification text describing the desired change, and a document gallery of candidate product images.

**Output**: A ranked list of candidate products from the gallery, or binary relevance labels for each candidate to compute top-k recall.

## Scoring recipe

```python
def compute_recall_at_k(predictions, gold_indices, k=5):
    top_k = predictions[:k]
    hits = sum(1 for idx in top_k if idx in gold_indices)
    return hits / len(gold_indices)
```

## Common pitfalls

- Using single-image queries instead of multi-view inputs, which the paper notes is fundamentally inadequate for this task.
- Relying on post-hoc aggregation (MeanPool/MaxSim) instead of native joint encoding for models that support it, as averaging can dilute view-specific details.
- Failing to restrict queries to short modification text only, which the authors emphasize reflects realistic user scenarios.

## Evidence (verbatim from paper)

> We evaluate using short modification text only, as it reflects the realistic user query scenario (§3.4), and report Recall@5 and Recall@10.

## Citation

```bibtex
@misc{yuan2026fashionmv,
  title={FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data},
  author={Yuan et al. (2026)},
  year={2026},
  note={arXiv:2604.10297}
}
```

- arXiv: 2604.10297