Easyvideor1 Eval

---
name: easyvideor1-eval
description: Evaluates the video understanding and reasoning capabilities of multimodal language models after reinforcement learning training. It probes performance across general video comprehension, long-context video understanding, complex reasoning, and STEM knowledge tasks using a standardized greedy decoding protocol. Use when the user wants to benchmark on Video-MME, MVBench, TempCompass, LVBench, LongVideoBench, MLVU, Video-Holmes, MMVU, Video-MMMU, VideoMathQA, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.16893
  bibtex_key: qin2026easyvideor1
  confidence: high
---

# easyvideor1-eval

> EasyVideoR1: Easier RL for Video Understanding — Chuanyu Qin et al. (2026) (arXiv:2604.16893, 2026)

## What this evaluates

Evaluates the video understanding and reasoning capabilities of multimodal language models after reinforcement learning training. It probes performance across general video comprehension, long-context video understanding, complex reasoning, and STEM knowledge tasks using a standardized greedy decoding protocol.

## Datasets

- **Video-MME, MVBench, TempCompass, LVBench, LongVideoBench, MLVU, Video-Holmes, MMVU, Video-MMMU, VideoMathQA** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered questions across all benchmark instances. Calculated as (number of correct predictions / total number of predictions) * 100.

## Input / output format

**Input**: Video sequences sampled at 2 FPS with a maximum of 128 frames and a per-frame pixel budget of 262,144, paired with text prompts. Image inputs use a separate pixel budget of 1,048,576.

**Output**: Free-form text responses generated via greedy decoding.

## Scoring recipe

```python
correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if pred.strip() == gold.strip():
        correct += 1
    total += 1
return (correct / total) * 100
```

## Common pitfalls

- Using non-greedy decoding during evaluation, as the protocol explicitly mandates greedy decoding for all benchmarks.
- Ignoring the specified video preprocessing parameters (2 FPS sampling, max 128 frames, 262,144 pixels/frame), which significantly affect model performance and throughput comparisons.

## Evidence (verbatim from paper)

> EasyVideoR1 training improves the average accuracy from 62.1 to 64.4 (+2.3), demonstrating that the framework’s end-to-end pipeline—from data loading through reward computation to policy update—functions correctly and effectively.

## Citation

```bibtex
@misc{qin2026easyvideor1,
  title={EasyVideoR1: Easier RL for Video Understanding},
  author={Chuanyu Qin et al. (2026)},
  year={2026},
  note={arXiv:2604.16893}
}
```

- arXiv: 2604.16893