Rationalrewards Eval

---
name: rationalrewards-eval
description: Evaluates a reasoning-based reward model's ability to produce human-aligned preference judgments and optimize visual generation models via reinforcement learning and test-time prompt refinement. Use when the user wants to benchmark on Multimodal Reward Bench 2 (MMRB2), EditReward Bench, GenAI-Bench, ImgEdit-Bench, GEdit-Bench-EN, UniGen (UniGenBench++), PICA-Bench, or asks about evaluating this task. Reports pairwise comparison accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.11626
  bibtex_key: [REDACTED]
  confidence: high
---

# rationalrewards-eval

> RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time — Haozhe Wang et al. (arXiv:2604.11626, 2026)

## What this evaluates

Evaluates a reasoning-based reward model's ability to produce human-aligned preference judgments and optimize visual generation models via reinforcement learning and test-time prompt refinement.

## Datasets

- **Multimodal Reward Bench 2 (MMRB2)** — total ?; splits: T2I (-1), Edit (-1)
- **EditReward Bench** — total ?; splits: Edit (-1)
- **GenAI-Bench** — total ?; splits: T2I (-1), Edit (-1)
- **ImgEdit-Bench** — total ?; splits: Edit (-1)
- **GEdit-Bench-EN** — total ?; splits: Edit (-1)
- **UniGen (UniGenBench++)** — total ?; splits: T2I (-1)
- **PICA-Bench** — total ?; splits: Edit (-1)

## Metrics

- `pairwise comparison accuracy` **(primary)** — range: percent
  - Percentage of correctly predicted preference labels (or ranked order) compared to human annotations across benchmark splits.
- `overall benchmark score` — range: percent
  - Aggregated score across subcategories (e.g., Action, Attribute, Layout, etc.) reported per benchmark protocol.

## Input / output format

**Input**: User prompt and generated image (for preference evaluation); user prompt and source image (for editing tasks).

**Output**: Structured multi-dimensional critiques with natural language refinement suggestions and dimension scores (threshold 3.0); or scalar/ranked preference labels for baseline comparison.

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels, benchmark_protocol):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    accuracy = (correct / len(gold_labels)) * 100
    overall = sum(benchmark_protocol.compute_subcategory_scores(predictions)) / len(benchmark_protocol.subcategories)
    return {'pairwise_accuracy': accuracy, 'overall_score': overall}
```

## Common pitfalls

- Confusing direct SFT distillation from a large teacher with the PARROT preference-anchored rationalization pipeline.
- Assuming test-time prompt tuning replaces RL entirely, whereas the paper shows they are complementary and prompt tuning often matches or exceeds RL only after RL or on specific benchmarks.
- Overlooking that training data uses raw preference pairs without explanations, requiring the PARROT pipeline to generate rationales before evaluation.

## Evidence (verbatim from paper)

> We report pairwise comparison accuracy on three established benchmarks: Multimodal Reward Bench 2 (Hu et al., 2025) and GenAI-Bench (Jiang et al., 2024) and EditReward Bench (Wu et al., 2025e) for both text- and image-to-image generation.

## Citation

```bibtex
@misc{wang2026rationalrewards,
  title={RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time},
  author={Haozhe Wang et al.},
  year={2026},
  note={arXiv:2604.11626}
}
```

- arXiv: 2604.11626