Imgedit Bench Eval

---
name: imgedit-bench-eval
description: Evaluates text-and-image-to-image editing capabilities, including addition, removal, replacement, motion change, style transfer, background change, object extraction, and hybrid edits. It tests the model's capacity to modify existing images according to natural language instructions while preserving unedited regions. Use when the user wants to benchmark on ImgEdit-Bench, or asks about evaluating this task. Reports ImgEdit-Bench.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.18095
  bibtex_key: chen2025sharegpt4oimage
  confidence: high
---

# imgedit-bench-eval

> ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation — Chen et al. (2025) (arXiv:2506.18095, 2025)

## What this evaluates

Evaluates text-and-image-to-image editing capabilities, including addition, removal, replacement, motion change, style transfer, background change, object extraction, and hybrid edits. It tests the model's capacity to modify existing images according to natural language instructions while preserving unedited regions.

## Datasets

- **ImgEdit-Bench** — total ?; splits: test (-1)

## Metrics

- `ImgEdit-Bench` **(primary)** — range: other
  - Average score across 8 edit categories (Addition, Removal, Replacement, Motion Change, Style Transfer, Background Change, Object Extraction, Hybrid Edit).

## Input / output format

**Input**: Source image paired with a text instruction specifying the edit type and target modification.

**Output**: Edited image.

## Scoring recipe

```python
scores = []
for img, instruction in dataset:
    edited_img = model.edit(img, instruction)
    score = judge.score(edited_img, img, instruction)
    scores.append(score)
return mean(scores)
```

## Common pitfalls

- Edit categories often overlap, making isolated evaluation challenging.
- The exact scoring scale (e.g., 1-5) is not explicitly stated in the paper excerpt.

## Evidence (verbatim from paper)

> Table 3: Evaluation of text-and-image-to-image generation ability on ImgEdit-Bench.

## Citation

```bibtex
@misc{chen2025sharegpt4oimage,
  title={ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation},
  author={Chen et al. (2025)},
  year={2025},
  note={arXiv:2506.18095}
}
```

- arXiv: 2506.18095