Treebench Eval

---
name: treebench-eval
description: Evaluates visual grounded reasoning by requiring models to precisely localize target objects in cluttered scenes and perform second-order reasoning about their interactions. It measures both the correctness of the final answer and the spatial accuracy of the traceable bounding box evidence. Use when the user wants to benchmark on TreeBench, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.07999
  bibtex_key: wang2025traceable
  confidence: high
---

# treebench-eval

> Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology — Wang et al. (2025) (arXiv:2507.07999, 2025)

## What this evaluates

Evaluates visual grounded reasoning by requiring models to precisely localize target objects in cluttered scenes and perform second-order reasoning about their interactions. It measures both the correctness of the final answer and the spatial accuracy of the traceable bounding box evidence.

## Datasets

- **TreeBench** — total ?; splits: Perception (-1), Reasoning (-1); repo https://github.com/Haochen-Wang409/TreeVGR

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly answered multiple-choice questions.
- `mIoU` — range: [0, 1]
  - Mean Intersection over Union between predicted and ground-truth bounding boxes across all questions, measuring localization precision.

## Input / output format

**Input**: A single image paired with a natural language question requiring object identification, spatial reasoning, or interaction analysis.

**Output**: A sequence of bounding boxes (traceable evidence) followed by a final answer or reasoning text.

## Scoring recipe

```python
def evaluate(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p['answer'] == g['answer'])
    acc = correct / len(gold)
    ious = []
    for p, g in zip(predictions, gold):
        pred_boxes = p['boxes']
        gt_boxes = g['boxes']
        iou_scores = compute_pairwise_iou(pred_boxes, gt_boxes)
        ious.append(mean(iou_scores))
    mIoU = mean(ious)
    return {'Accuracy': acc, 'mIoU': mIoU}
```

## Common pitfalls

- Models may output excessive candidate bounding boxes to maximize IoU recall, leading to repetition and failure to produce a final answer.
- High localization precision (mIoU) does not automatically translate to high reasoning accuracy, as complex reasoning requires second-order cognitive capabilities beyond spatial grounding.

## Evidence (verbatim from paper)

> Importantly, for visual grounded reasoning models, our traceable evaluation demonstrates a positive correlation between localization precision and the overall performance, as illustrated in Table[2]. This positive correlation between precise localization (mIoU) and overall performance is evident in the progressive improvement from DeepEyes-7B to Pixel-Reasoner-7B to our final TreeVGR-7B. As mIoU increases, the overall scores rise correspondingly, with TreeVGR-7B achieving the highest mIoU and strongest overall performance at the same time.

## Citation

```bibtex
@misc{wang2025traceable,
  title={Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2507.07999}
}
```

- arXiv: 2507.07999