Curriculum Dpo++ Eval

---
name: curriculum-dpo++-eval
description: Evaluates text-to-image generation models on their ability to align generated images with text prompts, produce visually appealing outputs, and match human preferences. It tests the effectiveness of curriculum-based fine-tuning strategies on standard generative benchmarks. Use when the user wants to benchmark on D1 (Black-ICLR-2024), D2 (DrawBench), D3 (Pick-a-Pic), or asks about evaluating this task. Reports Text Alignment.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.13055
  bibtex_key: [REDACTED]
  confidence: high
---

# curriculum-dpo++-eval

> Curriculum-DPO++: Direct Preference Optimization via Data and Model Curricula for Text-to-Image Generation — Florinel-Alin Croitoru et al. (arXiv:2602.13055, 2026)

## What this evaluates

Evaluates text-to-image generation models on their ability to align generated images with text prompts, produce visually appealing outputs, and match human preferences. It tests the effectiveness of curriculum-based fine-tuning strategies on standard generative benchmarks.

## Datasets

- **D1 (Black-ICLR-2024)** — total 74250; splits: train (67500), test (6750)
- **D2 (DrawBench)** — total 110000; splits: train (100000), test (10000)
- **D3 (Pick-a-Pic)** — total 150000; splits: test (500)

## Metrics

- `Text Alignment` **(primary)** — range: [-1, 1]
  - Cosine similarity between Sentence-BERT embeddings of the original prompt and a caption generated by LLaVA from the image.
- `Aesthetics` — range: other
  - Score from the LAION Aesthetics Predictor, a linear model based on CLIP representations.
- `Human Preference` — range: other
  - Score from HPSv2, a CLIP model fine-tuned on human-ranked image pairs.

## Input / output format

**Input**: Text prompts (SVO templates, DrawBench prompts, or Pick-a-Pic test prompts)

**Output**: Generated images (256x256 for SD, 768x768 for LCM)

## Scoring recipe

```python
def compute_metrics(prompts, images):
    scores = {'text_alignment': [], 'aesthetics': [], 'human_preference': []}
    for prompt, image in zip(prompts, images):
        caption = llava.generate_caption(image)
        scores['text_alignment'].append(cosine_similarity(sbert(prompt), sbert(caption)))
        scores['aesthetics'].append(laion_aesthetics.predict(image))
        scores['human_preference'].append(hpsv2.predict(image))
    return {k: np.mean(v) for k, v in scores.items()}
```

## Common pitfalls

- Using different reward models or generation settings across methods, which the paper explicitly warns against to ensure fair comparison.
- Confusing the reward-model-free training strategy (which uses prompt perturbation/masking to infer pair difficulty) with the evaluation metrics themselves.

## Evidence (verbatim from paper)

> To measure text alignment, we compute the cosine similarity between Sentence-BERT embeddings of the original prompt and a caption generated by LLaVA. For visual appeal, we utilize the LAION Aesthetics Predictor, a linear model based on CLIP representations. Finally, human preference is estimated using HPSv2, a CLIP model fine-tuned on human-ranked image pairs.

## Citation

```bibtex
@misc{croitoru2026curriculumdpo++,
  title={Curriculum-DPO++: Direct Preference Optimization via Data and Model Curricula for Text-to-Image Generation},
  author={Florinel-Alin Croitoru et al.},
  year={2026},
  note={arXiv:2602.13055}
}
```

- arXiv: 2602.13055