Refcoco Grounding Eval

---
name: refcoco-grounding-eval
description: Evaluates fine-grained visual grounding and spatial reasoning by measuring how accurately a multimodal model can localize regions in images corresponding to given referring expressions under varying linguistic and spatial conditions. Use when the user wants to benchmark on RefCOCO, RefCOCO+, RefCOCOg, or asks about evaluating this task. Reports IoU@50 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.12633
  bibtex_key: tao2025dig
  confidence: high
---

# refcoco-grounding-eval

> DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model — Tao et al. (2025) (arXiv:2512.12633, 2025)

## What this evaluates

Evaluates fine-grained visual grounding and spatial reasoning by measuring how accurately a multimodal model can localize regions in images corresponding to given referring expressions under varying linguistic and spatial conditions.

## Datasets

- **RefCOCO** — total ?; splits: val (-1), testA (-1), testB (-1)
- **RefCOCO+** — total ?; splits: val (-1), testA (-1), testB (-1)
- **RefCOCOg** — total ?; splits: val (-1), test (-1)

## Metrics

- `IoU@50 accuracy` **(primary)** — range: [0, 1]
  - Percentage of correctly localized regions where the Intersection over Union (IoU) between the predicted bounding box and the ground truth box is ≥ 0.5.
- `valavg / testAavg / testBavg` — range: [0, 1]
  - Average accuracy across IoU thresholds of 50, 75, and 95 for the validation, testA, or testB splits respectively.

## Input / output format

**Input**: An image and a referring expression (text query describing a target object/region).

**Output**: Bounding box coordinates (typically [x_min, y_min, x_max, y_max] normalized to [0, 1] or in pixel space).

## Scoring recipe

```python
def compute_iou(box1, box2):
    x1 = max(box1[0], box2[0])
    y1 = max(box1[1], box2[1])
    x2 = min(box1[2], box2[2])
    y2 = min(box1[3], box2[3])
    inter = max(0, x2 - x1) * max(0, y2 - y1)
    area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
    area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])
    return inter / (area1 + area2 - inter)

def score(predictions, golds, threshold=0.5):
    correct = sum(1 for pred, gold in zip(predictions, golds) if compute_iou(pred, gold) >= threshold)
    return correct / len(golds)
```

## Common pitfalls

- RefCOCO has two distinct test sets (testA from Flickr30k, testB from COCO); results must be reported separately.
- IoU thresholds of 50, 75, and 95 are standard; reporting only one may misrepresent performance.
- Coordinate format (normalized vs. absolute pixels) must match the ground truth scale to avoid incorrect IoU calculations.

## Evidence (verbatim from paper)

> Results on referring expression comprehension datasets RefCOCO, RefCOCO+, and RefCOCOg. Incorporating DiG into Qwen3-VL models consistently improves localization accuracy across different IoU thresholds, demonstrating stronger fine-grained spatial grounding and generalization ability.

## Citation

```bibtex
@misc{tao2025dig,
  title={DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model},
  author={Tao et al. (2025)},
  year={2025},
  note={arXiv:2512.12633}
}
```

- arXiv: 2512.12633