Videop2r Eval

---
name: videop2r-eval
description: Evaluates large video language models on their ability to perceive visual details and perform multi-step reasoning over video content. It measures how well models decompose video understanding into distinct perception and reasoning stages across multiple benchmarks. Use when the user wants to benchmark on VSI-Bench, VideoMMMU, MMVU, VCR, MV, TempCom, VideoMME, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.11113
  bibtex_key: jiang2025videop2r
  confidence: high
---

# videop2r-eval

> VIDEOP2R: Video Understanding from Perception to Reasoning — Jiang et al. (arXiv:2511.11113, 2025)

## What this evaluates

Evaluates large video language models on their ability to perceive visual details and perform multi-step reasoning over video content. It measures how well models decompose video understanding into distinct perception and reasoning stages across multiple benchmarks.

## Datasets

- **VSI-Bench** — total ?; splits: test (-1)
- **VideoMMMU** — total ?; splits: test (-1)
- **MMVU** — total ?; splits: test (-1)
- **VCR** — total ?; splits: test (-1)
- **MV** — total ?; splits: test (-1)
- **TempCom** — total ?; splits: test (-1)
- **VideoMME** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered questions out of the total number of questions in each benchmark.

## Input / output format

**Input**: Video input (frames/clips) paired with a text-based question requiring visual perception and reasoning.

**Output**: Text response containing a step-by-step reasoning trace enclosed in <think> tags, followed by a final answer.

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p.strip() == g.strip())
    return (correct / len(gold)) * 100
```

## Common pitfalls

- Performance on domain-specific benchmarks like MMVU drops if the training data lacks factual knowledge (e.g., chemistry).
- Models may exhibit 'Think-Answer Mismatch', where reasoning traces are unfaithful to the actual decision process but still yield correct answers, leading to reward hacking in standard RL setups.
- Length rewards can be counter-productive for questions requiring long, fine-grained descriptions.

## Evidence (verbatim from paper)

> Compared with prior video RFT approaches, VideoP2R achieves highly competitive performance across seven benchmarks, setting SotA results on six of them and ranking second on the remaining one. In contrast to previous RFT methods that often bring improvement on specific datasets (e.g., Video-R1 ranks second on VSI-Bench, while VideoRFT is SotA on MMVU but last on MVBench), VideoP2R delivers consistent gains across all benchmarks, surpassing the previous SotA by 1.3% in average accuracy.

## Citation

```bibtex
@misc{jiang2025videop2r,
  title={VIDEOP2R: Video Understanding from Perception to Reasoning},
  author={Jiang et al.},
  year={2025},
  note={arXiv:2511.11113}
}
```

- arXiv: 2511.11113