Tablevision Eval

---
name: tablevision-eval
description: Probes multimodal large language models' ability to perform spatially grounded reasoning over complex hierarchical tables. It specifically evaluates performance degradation across three cognitive levels (Perception, Reasoning, Analysis) and measures how explicit spatial anchoring mitigates perceptual overload and spatial attention failure. Use when the user wants to benchmark on TableVision, or asks about evaluating this task. Reports exact-match Accuracy (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.03660
  bibtex_key: chen2026tablevision
  confidence: high
---

# tablevision-eval

> TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tables — Chen et al. (2026) (arXiv:2604.03660, 2026)

## What this evaluates

Probes multimodal large language models' ability to perform spatially grounded reasoning over complex hierarchical tables. It specifically evaluates performance degradation across three cognitive levels (Perception, Reasoning, Analysis) and measures how explicit spatial anchoring mitigates perceptual overload and spatial attention failure.

## Datasets

- **TableVision** — total 6799; splits: test (1301), train (-1), val (-1)

## Metrics

- `exact-match Accuracy (%)` **(primary)** — range: percent
  - Calculated as the percentage of instances where the model's predicted answer exactly matches the ground-truth answer, aggregated across 13 fine-grained task categories spanning L1 (Perception), L2 (Reasoning), and L3 (Analysis) cognitive levels.

## Input / output format

**Input**: Image of a complex hierarchical table paired with a natural language query/question requiring spatial grounding and logical reasoning.

**Output**: Direct text answer (for zero-shot baseline evaluation) or structured chain-of-thought including bounding box coordinates followed by the final answer (for the proposed two-stage framework).

## Scoring recipe

```python
correct = 0
for pred, gold in zip(predictions, golds):
    if pred.strip().lower() == gold.strip().lower():
        correct += 1
accuracy = (correct / len(golds)) * 100
```

## Common pitfalls

- Models often suffer from perceptual overload in dense tables, causing spatial attention failure and operating on incorrect semantic content.
- Zero-shot evaluation may mask underlying reasoning capabilities; providing ground-truth bounding boxes (Oracle probing) reveals a 20.7% accuracy surge, indicating spatial disorientation is the primary bottleneck rather than logical failure.
- Fine-tuning for complex L2/L3 tasks can cause performance regression on simple L1 perception tasks due to reasoning overhead and task interference.

## Evidence (verbatim from paper)

> We report the exact-match Accuracy (%) across the 13 fine-grained task categories spanning cognitive levels L1 to L3.

## Citation

```bibtex
@misc{chen2026tablevision,
  title={TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tables},
  author={Chen et al. (2026)},
  year={2026},
  note={arXiv:2604.03660}
}
```

- arXiv: 2604.03660