Vbench T2v

---
name: vbench-t2v
description: Evaluates text-to-video generation quality and semantic alignment across dimensions like human action, scene composition, object consistency, and aesthetic quality. Use when the user wants to benchmark on VBench, or asks about evaluating this task. Reports VBench Overall.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.13030
  bibtex_key: han2026generativerefinement
  confidence: high
---

# vbench-t2v

> Generative Refinement Networks for Visual Synthesis — Jian Han et al. (arXiv:2604.13030, 2026)

## What this evaluates

Evaluates text-to-video generation quality and semantic alignment across dimensions like human action, scene composition, object consistency, and aesthetic quality.

## Datasets

- **VBench** — total ?; splits: test (-1)

## Metrics

- `VBench Overall` **(primary)** — range: percent
  - Composite score aggregating Human Action, Scene, Multiple Objects, Appearance Style, Quality Score, and Semantic Score. Reported as percentage.

## Input / output format

**Input**: Text prompt describing video content

**Output**: Video clip (2-10 seconds, 256x256 to 480p)

## Scoring recipe

```python
videos = []
for prompt in vbench_prompts:
    videos.append(model.generate(prompt))
scores = vbench.evaluate(videos)
overall = mean(scores.dimensions)
```

## Common pitfalls

- Scores are computed on videos generated after prompt rewriting.
- Automated VBench metrics may not fully capture temporal coherence or human preference.

## Evidence (verbatim from paper)

> GRN exhibits superior performance in generating videos from textual prompts. When benchmarked against contemporary diffusion and flow-based models—including AnimateDiff-V2 *[animatediff]*, VideoCraft-2.0 *[videocrafter]*, OpenSora V1.2 *[opensora]*, Show-1 *[show-1]*, and CogVideoX-5B *[cogvideox]*—GRN achieves significantly higher scores across quality, semantic, and overall scores.

## Citation

```bibtex
@misc{han2026generativerefinement,
  title={Generative Refinement Networks for Visual Synthesis},
  author={Jian Han et al.},
  year={2026},
  note={arXiv:2604.13030}
}
```

- arXiv: 2604.13030