Multimodal Reward Benchmarks Eval

---
name: multimodal-reward-benchmarks-eval
description: Evaluates multimodal reward models on preference ranking tasks across image and video domains, measuring how well they score or rank candidate responses compared to ground-truth preferences. It compares multi-response scoring against single-response baselines and generative judges, while also assessing inference efficiency and downstream policy optimization stability. Use when the user wants to benchmark on VL-RewardBench, Multimodal RewardBench, MM-RLHF RewardBench, MR2Bench-Image, VideoRewardBench, MR2Bench-Video, or asks about evaluating this task. Reports pairwise accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.10966
  bibtex_key: [REDACTED]
  confidence: high
---

# multimodal-reward-benchmarks-eval

> You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass — Yang et al. (2026) (arXiv:2604.10966, 2026)

## What this evaluates

Evaluates multimodal reward models on preference ranking tasks across image and video domains, measuring how well they score or rank candidate responses compared to ground-truth preferences. It compares multi-response scoring against single-response baselines and generative judges, while also assessing inference efficiency and downstream policy optimization stability.

## Datasets

- **VL-RewardBench** — total ?; splits: test (-1)
- **Multimodal RewardBench** — total ?; splits: test (-1)
- **MM-RLHF RewardBench** — total ?; splits: test (-1)
- **MR2Bench-Image** — total ?; splits: test (-1)
- **VideoRewardBench** — total ?; splits: test (-1)
- **MR2Bench-Video** — total ?; splits: test (-1)

## Metrics

- `pairwise accuracy` **(primary)** — range: [0, 100] percent
  - Percentage of correctly predicted preferred responses in pairwise comparisons. Reported as macro pairwise accuracy for VL-RewardBench and VideoRewardBench, and standard pairwise accuracy for Multimodal RewardBench and MM-RLHF RewardBench.
- `best-of-4 accuracy` — range: [0, 100] percent
  - Percentage of samples where the model correctly selects the top-ranked response from a set of 4 candidates.

## Input / output format

**Input**: Image or video context with N candidate responses concatenated using separator tokens.

**Output**: N scalar reward scores (one per response) or a predicted best response index.

## Scoring recipe

```python
def compute_accuracy(predictions, gold, metric_type):
    correct = 0
    for pred, gold_label in zip(predictions, gold):
        if metric_type == 'pairwise':
            if pred == gold_label: correct += 1
        elif metric_type == 'best-of-4':
            if pred == gold_label: correct += 1
    return (correct / len(gold)) * 100
```

## Common pitfalls

- Pairwise accuracy and best-of-4 accuracy use different evaluation protocols and should not be averaged without noting the difference.
- Single-response Bradley-Terry scoring requires N forward passes per sample, while multi-response scoring uses one pass; comparing them requires equalizing compute or explicitly reporting the N× speedup.
- Response order bias can affect multi-response models if not shuffled during training or evaluated across permutations.

## Evidence (verbatim from paper)

> VL-RB: VL-RewardBench (macro pairwise acc.); MM-RB: Multimodal RewardBench (pairwise acc.); MMRLHF: MM-RLHF RewardBench (pairwise acc.); MR2B-I: MR2Bench-Image (best-of-4 acc.); VRB: VideoRewardBench (macro pairwise acc.); MR2B-V: MR2Bench-Video (best-of-4 acc.).

## Citation

```bibtex
@misc{yang2026youonlyjudgeonce,
  title={You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass},
  author={Yang et al. (2026)},
  year={2026},
  note={arXiv:2604.10966}
}
```

- arXiv: 2604.10966