Geneval T2i

---
name: geneval-t2i
description: Evaluates text-to-image generation by measuring how accurately models follow complex prompts with multiple objects, attributes, and spatial constraints. Use when the user wants to benchmark on GenEval, or asks about evaluating this task. Reports GenEval Overall.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.13030
  bibtex_key: han2026generativerefinement
  confidence: high
---

# geneval-t2i

> Generative Refinement Networks for Visual Synthesis — Jian Han et al. (arXiv:2604.13030, 2026)

## What this evaluates

Evaluates text-to-image generation by measuring how accurately models follow complex prompts with multiple objects, attributes, and spatial constraints.

## Datasets

- **GenEval** — total ?; splits: test (-1)

## Metrics

- `GenEval Overall` **(primary)** — range: [0, 1]
  - Average accuracy across sub-tasks: Single Object, Two Object, Count, Colors, Position, and Color Attributes. Ranges from 0 to 1.

## Input / output format

**Input**: Text prompt specifying object count, colors, and spatial relationships

**Output**: Generated image

## Scoring recipe

```python
scores = []
for prompt in geneval_prompts:
    img = model.generate(prompt)
    match = detector.check_constraints(img, prompt)
    scores.append(match)
overall = mean(scores)
```

## Common pitfalls

- Prompt rewriting is applied before generation, which can artificially boost scores.
- Evaluation relies on an external object/attribute detector that may have its own failure modes.

## Evidence (verbatim from paper)

> our model, augmented with a re-writer, achieves an overall score of 0.76 on the GenEval benchmark *[ghosh2024geneval]*.

## Citation

```bibtex
@misc{han2026generativerefinement,
  title={Generative Refinement Networks for Visual Synthesis},
  author={Jian Han et al.},
  year={2026},
  note={arXiv:2604.13030}
}
```

- arXiv: 2604.13030