Ris Lad Eval

---
name: ris-lad-eval
description: Evaluates referring image segmentation on low-altitude drone imagery, probing the model's ability to accurately localize and segment referred objects despite challenges like category drift (tiny objects) and object drift (dense same-category scenes). Use when the user wants to benchmark on RIS-LAD, or asks about evaluating this task. Reports oIoU, mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.20920
  bibtex_key: ye2025rislad
  confidence: high
---

# ris-lad-eval

> RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation — Ye et al. (2025) (arXiv:2507.20920, 2025)

## What this evaluates

Evaluates referring image segmentation on low-altitude drone imagery, probing the model's ability to accurately localize and segment referred objects despite challenges like category drift (tiny objects) and object drift (dense same-category scenes).

## Datasets

- **RIS-LAD** — total ?; splits: val (-1), test (-1); repo https://github.com/AHideoKuzeA/RIS-LAD-A-Benchmark-and-Model-for-Referring-Low-Altitude-Drone-Image-Segmentation

## Metrics

- `oIoU` **(primary)** — range: percent
  - Overall Intersection-over-Union: the average IoU across all pixels in the image between the predicted mask and the ground truth mask.
- `mIoU` **(primary)** — range: percent
  - Mean Intersection-over-Union: the average IoU across all object instances in the image.
- `P@X` — range: percent
  - Precision at IoU threshold X: the percentage of predicted masks that achieve an IoU ≥ X with the ground truth.

## Input / output format

**Input**: A low-altitude drone image paired with a referring text expression describing the target object.

**Output**: A binary segmentation mask indicating the pixels belonging to the referred object.

## Scoring recipe

```python
def compute_metrics(pred_masks, gt_masks, X):
    ious = []
    for pred, gt in zip(pred_masks, gt_masks):
        intersection = np.logical_and(pred, gt).sum()
        union = np.logical_or(pred, gt).sum()
        ious.append(intersection / union if union > 0 else 0.0)
    oIoU = np.mean(ious) * 100
    mIoU = np.mean(ious) * 100
    P_at_X = sum(1 for iou in ious if iou >= X) / len(ious) * 100
    return oIoU, mIoU, P_at_X
```

## Common pitfalls

- P@X may overestimate performance on small objects due to their ease of enclosure.
- Methods relying on frozen pre-trained encoders (e.g., CLIP) may suffer negative transfer due to domain gaps with drone imagery.
- Boundary overgeneralization can lead to high P@X but low oIoU/mIoU.

## Evidence (verbatim from paper)

> For metrics, we report Precision@0.5–0.9 (P@X), Overall Intersection-over-Union (oIoU), and Mean Intersection-over-Union (mIoU). While P@X highlights accurate predictions, it may overestimate performance on small objects due to their ease of enclosure. Thus, oIoU and mIoU are used as primary metrics to evaluate segmentation performance.

## Citation

```bibtex
@misc{ye2025rislad,
  title={RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation},
  author={Ye et al. (2025)},
  year={2025},
  note={arXiv:2507.20920}
}
```

- arXiv: 2507.20920