Dpg Bench Eval

---
name: dpg-bench-eval
description: Evaluates dense prompt following on multi-requirement prompts by decomposing them into dependency-structured VQA checks spanning entity presence, attributes, relations, and counts. Use when the user wants to benchmark on DPG-Bench, or asks about evaluating this task. Reports Overall.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.12163
  bibtex_key: [REDACTED]
  confidence: high
---

# dpg-bench-eval

> Nucleus-Image: Sparse MoE for Image Generation — Akiti et al. (2026) (arXiv:2604.12163, 2026)

## What this evaluates

Evaluates dense prompt following on multi-requirement prompts by decomposing them into dependency-structured VQA checks spanning entity presence, attributes, relations, and counts.

## Datasets

- **DPG-Bench** — total 1065; splits: test (1065)

## Metrics

- `Overall` **(primary)** — range: [0, 100]
  - Mean score across five subcategories: Global, Entity, Attribute, Relation, and Other.

## Input / output format

**Input**: Text prompt containing multiple dense requirements and constraints.

**Output**: Generated image at 1024x1024 resolution, 50 inference steps, CFG scale 8.0.

## Scoring recipe

```python
scores = []
for prompt in prompts:
    img = model.generate(prompt, steps=50, cfg=8.0, res=1024)
    checks = decompose_prompt_into_vqa(prompt)
    correct = sum(vqa_model.verify(img, check) for check in checks)
    scores.append(correct / len(checks))
return mean(scores) * 100
```

## Common pitfalls

- Global consistency scores are often lower than fine-grained entity/attribute scores, indicating holistic scene coherence is harder to achieve.
- Evaluation requires careful decomposition of prompts into structured VQA checks to avoid missing implicit constraints.

## Evidence (verbatim from paper)

> Nucleus-Image achieves the highest overall score of 88.79, narrowly outperforming Qwen-Image (88.32) and Seedream 3.0 (88.27).

## Citation

```bibtex
@misc{akiti2026nucleusimage,
  title={Nucleus-Image: Sparse MoE for Image Generation},
  author={Akiti et al. (2026)},
  year={2026},
  note={arXiv:2604.12163}
}
```

- arXiv: 2604.12163