Remindviewbench Eval

---
name: remindviewbench-eval
description: Evaluates vision-language models' ability to perform multi-view spatial reasoning, including relative direction, relative distance, cross-view consistency, and perspective-taking. It probes whether models can maintain geometric coherence and integrate information across multiple camera viewpoints to answer questions about procedurally generated indoor scenes. Use when the user wants to benchmark on ReMindView-Bench, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.02340
  bibtex_key: xue2025remindviewbench
  confidence: high
---

# remindviewbench-eval

> Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective — Xue et al. (2025) (arXiv:2512.02340, 2025)

## What this evaluates

Evaluates vision-language models' ability to perform multi-view spatial reasoning, including relative direction, relative distance, cross-view consistency, and perspective-taking. It probes whether models can maintain geometric coherence and integrate information across multiple camera viewpoints to answer questions about procedurally generated indoor scenes.

## Datasets

- **ReMindView-Bench** — total 50000; splits: test (-1); repo https://github.com/pittisl/ReMindView-Bench

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Standard multiple-choice accuracy: the fraction of questions where the model's predicted option exactly matches the ground-truth answer derived from scene metadata.

## Input / output format

**Input**: One or more rendered images of an indoor scene (single-view or multi-view) paired with a natural language question about spatial relationships, distances, or object locations.

**Output**: A single selected option from a predefined set of candidate answers (multiple-choice format).

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for pred, gold in zip(predictions, gold_labels) if pred == gold)
    return correct / len(gold_labels)
```

## Common pitfalls

- Models often succeed on single-view questions but degrade sharply when cross-view integration is required.
- Perspective-taking queries require shifting the reasoning frame to an object-centered viewpoint, which many VLMs fail to maintain consistently.
- Geometric metadata is used for ground truth, but models rely on visual cues that may be ambiguous or occluded across views.

## Evidence (verbatim from paper)

> ReMindView-Bench consists of $>50,000$ multi-choice VQA pairs derived from 100 procedurally generated indoor scenes, and is constructed through a fully automated and physically grounded pipeline. ... For each query, candidate answers are drawn from the visible objects in the scene, and the correct answer is computed from the metadata.

## Citation

```bibtex
@misc{xue2025remindviewbench,
  title={Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective},
  author={Xue et al. (2025)},
  year={2025},
  note={arXiv:2512.02340}
}
```

- arXiv: 2512.02340