Video To C Eval

---
name: video-to-c-eval
description: Evaluates fine-grained video understanding, spatio-temporal reasoning, and hallucination mitigation in multimodal large language models by testing their ability to locate key visual cues and answer questions across diverse video benchmarks. Use when the user wants to benchmark on VSI-Bench, VideoMMMU, MMVU, MVBench, TempCompass, VideoMME, VideoHallucer, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.20473
  bibtex_key: tan2026videotoc
  confidence: high
---

# video-to-c-eval

> Video-ToC: Video Tree-of-Cue Reasoning — Tan et al. (2026) (arXiv:2604.20473, 2026)

## What this evaluates

Evaluates fine-grained video understanding, spatio-temporal reasoning, and hallucination mitigation in multimodal large language models by testing their ability to locate key visual cues and answer questions across diverse video benchmarks.

## Datasets

- **VSI-Bench** — total ?; splits: test (-1)
- **VideoMMMU** — total ?; splits: test (-1)
- **MMVU** — total ?; splits: test (-1)
- **MVBench** — total ?; splits: test (-1)
- **TempCompass** — total ?; splits: test (-1)
- **VideoMME** — total ?; splits: test (-1)
- **VideoHallucer** — total ?; splits: test (-1)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly answered questions after extracting the model's prediction from the <answer> tags and comparing it to the ground truth label. Average accuracy is computed across all benchmarks.

## Input / output format

**Input**: Video frames (uniformly sampled 16, 32, or 64 frames at 128x28x28 or 256x28x28 resolution) paired with a text question. A unified prompt is used, with task instructions specifying the expected output format based on question type (multiple-choice or numerical/regression).

**Output**: Text response containing the model's reasoning and final answer enclosed in <answer> </answer> tags. For multiple-choice questions, only the option letter is provided inside the tags; for numerical questions, the value is provided.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = 0
    for pred, gold in zip(predictions, gold_labels):
        ans = extract_between_tags(pred, '<answer>', '</answer>')
        if ans is not None and normalize(ans) == normalize(gold):
            correct += 1
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- VideoMME must be evaluated without subtitle assistance to match the reported protocol.
- MMVU evaluation strictly uses the multiple-choice question subset, not the full dataset.
- Parsing the <answer> tags is required; models that output answers outside these tags will be scored as incorrect or fail to parse.

## Evidence (verbatim from paper)

> TABLE I: Accuracy comparison on three video reasoning benchmarks and three video general benchmarks. “Avg.” denotes average accuracy of the six benchmarks.

## Citation

```bibtex
@misc{tan2026videotoc,
  title={Video-ToC: Video Tree-of-Cue Reasoning},
  author={Tan et al. (2026)},
  year={2026},
  note={arXiv:2604.20473}
}
```

- arXiv: 2604.20473