Points Long Video Image Eval

---
name: points-long-video-image-eval
description: Evaluates multimodal large language models on fine-grained image understanding and long-form video comprehension tasks. It measures the trade-off between visual token compression efficiency and task accuracy across diverse benchmarks. Use when the user wants to benchmark on MVBench, Video-MME, MLVU, LongVideoBench, MMBench, MMMU_val, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.11627
  bibtex_key: [REDACTED]
  confidence: high
---

# points-long-video-image-eval

> POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs — Haicheng Wang et al. (arXiv:2604.11627, 2026)

## What this evaluates

Evaluates multimodal large language models on fine-grained image understanding and long-form video comprehension tasks. It measures the trade-off between visual token compression efficiency and task accuracy across diverse benchmarks.

## Datasets

- **MVBench** — total ?; splits: test (-1)
- **Video-MME** — total ?; splits: test (-1)
- **MLVU** — total ?; splits: test (-1)
- **LongVideoBench** — total ?; splits: test (-1)
- **MMBench** — total ?; splits: test (-1)
- **MMMU_val** — total ?; splits: val (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered questions across each benchmark. Calculated as (number of correct predictions / total predictions) * 100. Evaluated using benchmark-specific protocols via VLMEvalKit and lmms-eval.

## Input / output format

**Input**: Video sequences (typically 64 frames) or images paired with text prompts/questions. Visual tokens are dynamically compressed to 8, 16, or 32 per frame in Standby mode, or retained densely in Focus mode.

**Output**: Text responses answering the provided VQA prompts.

## Scoring recipe

```python
correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if pred.strip().lower() == gold.strip().lower():
        correct += 1
    total += 1
return (correct / total) * 100
```

## Common pitfalls

- Confusing Standby vs Focus mode token budgets (2.5%-10% vs ~100%) and their respective performance baselines.
- Assuming standard 64-frame evaluation applies to streaming benchmarks, which use variable frame counts (e.g., 248+8, 504+8).
- Overlooking that average scores are computed across heterogeneous benchmarks with different evaluation protocols (exact match, LLM-judge, etc.).

## Evidence (verbatim from paper)

> We evaluate on a wide range of video benchmarks, including Opencompass video leaderboard: VideoMME, Tempcompass, MVBench, MMBench-Video, MLVU, LongVideoBench, and other commonly used video benchmarks: MovieChat1K, CG-Bench, EgoSchema, TemporalBench, Activitynet-qa, LVBench and WorldSense. We use VLMEvalKit and lmms-eval for evaluation.

## Citation

```bibtex
@misc{wang2026pointslong,
  title={POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs},
  author={Haicheng Wang et al.},
  year={2026},
  note={arXiv:2604.11627}
}
```

- arXiv: 2604.11627