Vlm Interaction Reasoning Eval

---
name: vlm-interaction-reasoning-eval
description: Evaluates vision-language models on general visual understanding, spatial/relational reasoning, and specifically interactional reasoning in dynamic scenes using a suite of standard VQA and scene understanding benchmarks. Use when the user wants to benchmark on VQAv2, VizWiz, TextVQA, GQA, VSR, RealWorldQA, MMT-Bench, SEEDBench, A-Bench, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.09118
  bibtex_key: liang2025seeing
  confidence: high
---

# vlm-interaction-reasoning-eval

> Seeing Beyond the Scene: Enhancing Vision-Language Models with Interactional Reasoning — Dayong Liang et al. (2025) (arXiv:2505.09118, 2025)

## What this evaluates

Evaluates vision-language models on general visual understanding, spatial/relational reasoning, and specifically interactional reasoning in dynamic scenes using a suite of standard VQA and scene understanding benchmarks.

## Datasets

- **VQAv2** — total ?; splits: test (-1)
- **VizWiz** — total ?; splits: test (-1)
- **TextVQA** — total ?; splits: test (-1)
- **GQA** — total ?; splits: test (-1)
- **VSR** — total ?; splits: test (-1)
- **RealWorldQA** — total ?; splits: test (-1)
- **MMT-Bench** — total ?; splits: test (-1)
- **SEEDBench** — total ?; splits: test (-1)
- **A-Bench** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Exact-match accuracy: the percentage of predictions that exactly match the ground truth answer after lowercasing and stripping whitespace.

## Input / output format

**Input**: An image and a natural language question.

**Output**: A short text answer or phrase.

## Scoring recipe

```python
correct = 0
for pred, gold in zip(predictions, golds):
    if pred.strip().lower() == gold.strip().lower():
        correct += 1
return (correct / len(golds)) * 100
```

## Common pitfalls

- Data leakage is likely, as the training data includes LLaVA-v1.5-mixed-665k and OpenImages, which overlap with test benchmarks like VQAv2 and GQA.
- Exact-match scoring on VQA benchmarks is highly sensitive to minor phrasing differences and ignores semantic equivalence.
- The paper reports average performance across benchmarks but does not provide statistical significance testing or confidence intervals.

## Evidence (verbatim from paper)

> We carefully selected a diverse suite of benchmarks to comprehensively evaluate both general vision-language capabilities and specific interactional reasoning skills: General VL Understanding: VQAv2, VizWiz, and TextVQA... Spatial & Relational Understanding: GQA, VSR... Real-world Interaction Understanding: RealWorldQA, MMT-Bench... Compositional Reasoning: SEEDBench, A-Bench

## Citation

```bibtex
@misc{liang2025seeing,
  title={Seeing Beyond the Scene: Enhancing Vision-Language Models with Interactional Reasoning},
  author={Dayong Liang et al. (2025)},
  year={2025},
  note={arXiv:2505.09118}
}
```

- arXiv: 2505.09118