Gseval Pixel Grounding Eval

---
name: gseval-pixel-grounding-eval
description: Evaluates a model's ability to perform open-vocabulary, fine-grained pixel grounding by generating accurate segmentation masks from complex, long-form referring expressions across multiple granularities (stuff, part, multi-object, single-object). Use when the user wants to benchmark on GSEval, gRefCOCO, RefCOCOm, RefCOCO, RefCOCOg, or asks about evaluating this task. Reports cIoU / gIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2503.10596
  bibtex_key: hu2025groundingsuite
  confidence: high
---

# gseval-pixel-grounding-eval

> GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding — Hu et al. (2025) (arXiv:2503.10596, 2025)

## What this evaluates

Evaluates a model's ability to perform open-vocabulary, fine-grained pixel grounding by generating accurate segmentation masks from complex, long-form referring expressions across multiple granularities (stuff, part, multi-object, single-object).

## Datasets

- **GSEval** — total 3800; splits: test (-1)
- **gRefCOCO** — total ?; splits: test (-1)
- **RefCOCOm** — total ?; splits: test (-1)
- **RefCOCO** — total ?; splits: test (-1)
- **RefCOCOg** — total ?; splits: test (-1)

## Metrics

- `cIoU / gIoU` **(primary)** — range: [0, 1]
  - Intersection over Union (IoU) between the predicted segmentation mask and the ground-truth mask. The paper reports cIoU (centered IoU) for gRefCOCO and gIoU (generalized IoU) for RefCOCOm, with higher values indicating better pixel-level alignment.

## Input / output format

**Input**: An image paired with a natural language referring expression (text description).

**Output**: A binary segmentation mask (pixel-level prediction) corresponding to the object(s) described in the text.

## Scoring recipe

```python
def compute_iou(pred_mask, gt_mask):
    intersection = np.logical_and(pred_mask, gt_mask).sum()
    union = np.logical_or(pred_mask, gt_mask).sum()
    return intersection / union if union > 0 else 0.0
# Metric is averaged over all test instances.
```

## Common pitfalls

- Evaluating on multi-granular targets (stuff, part, multi-object, single-object) requires consistent mask generation across varying object complexities.
- Long-form referring expressions (averaging 16 words) demand fine-grained semantic understanding, making models prone to hallucination or misalignment without robust training data.
- Zero-shot evaluation on open-source methods must use public code and weights to ensure fair comparison, as noted in the figure captions.

## Evidence (verbatim from paper)

> The resulting GSEval benchmark (3,800 images) offers zero overlap with existing datasets and evaluates open-vocabulary, fine-grained, and complex scene grounding, enabling state-of-the-art performance (cIoU 68.9 on gRefCOCO, gIoU 55.3 on RefCOCOm) and setting new benchmarks for pixel grounding across multi-granular and open-domain scenarios.

## Citation

```bibtex
@misc{hu2025groundingsuite,
  title={GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding},
  author={Hu et al. (2025)},
  year={2025},
  note={arXiv:2503.10596}
}
```

- arXiv: 2503.10596