Refaerial Eval

---
name: refaerial-eval
description: Evaluates a model's ability to localize a target object in an aerial image based on a fine-grained natural language description. It probes cross-modal alignment, scale-invariant object detection, and handling of complex backgrounds with numerous distractors. Use when the user wants to benchmark on RefAerial, RefCOCO, RefCOCO+, RefCOCOg, or asks about evaluating this task. Reports mP (average P@0.5/0.6/0.7/0.8).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.20543
  bibtex_key: hu2026refaerial
  confidence: high
---

# refaerial-eval

> RefAerial: A Benchmark and Approach for Referring Detection in Aerial Images — Hu et al. (2026) (arXiv:2604.20543, 2026)

## What this evaluates

Evaluates a model's ability to localize a target object in an aerial image based on a fine-grained natural language description. It probes cross-modal alignment, scale-invariant object detection, and handling of complex backgrounds with numerous distractors.

## Datasets

- **RefAerial** — total ?; splits: test (-1)
- **RefCOCO** — total ?; splits: test (-1)
- **RefCOCO+** — total ?; splits: test (-1)
- **RefCOCOg** — total ?; splits: test (-1)

## Metrics

- `mP (average P@0.5/0.6/0.7/0.8)` **(primary)** — range: percent
  - Precision at IoU threshold $\theta$, denoted P@$\theta$. A prediction is correct if IoU(pred, gt) > $\theta$. mP is the arithmetic mean of P@0.5, P@0.6, P@0.7, and P@0.8.

## Input / output format

**Input**: Aerial image and a natural language referring expression.

**Output**: A single bounding box (e.g., [x_min, y_min, x_max, y_max]) localizing the target object.

## Scoring recipe

```python
def compute_precision(predictions, ground_truths, iou_threshold):
    correct = 0
    for pred_box, gt_box in zip(predictions, ground_truths):
        iou = calculate_iou(pred_box, gt_box)
        if iou > iou_threshold:  # strictly greater than
            correct += 1
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- IoU threshold comparison is strictly greater than (>) the threshold, not greater-than-or-equal-to (≥).
- Precision is computed per referring expression (1-to-1 mapping), not per image, as each expression corresponds to exactly one ground-truth box.
- Models must be re-trained or fine-tuned on the dataset; direct transfer of ground-view checkpoints causes severe performance drops due to aerial scale variability.

## Evidence (verbatim from paper)

> Following the standard protocol for referring detection in [[20](#bib.bib20 ""), [38](#bib.bib38 ""), [7](#bib.bib7 "")]*, we use precision (P@$\theta$) as the evaluation metric, where the prediction is deemed correct if its IoU with the ground-truth box is larger than threshold $\theta$. In detail, we utilize P@0.5, P@0.6, P@0.7, P@0.8, and mP (average pecision across these thresholds) except specially mentioned.

## Citation

```bibtex
@misc{hu2026refaerial,
  title={RefAerial: A Benchmark and Approach for Referring Detection in Aerial Images},
  author={Hu et al. (2026)},
  year={2026},
  note={arXiv:2604.20543}
}
```

- arXiv: 2604.20543