Skyscenes Aerial Seg Eval

---
name: skyscenes-aerial-seg-eval
description: Evaluates semantic segmentation models trained on synthetic aerial imagery for their ability to generalize to real-world UAV datasets and adapt to varying environmental conditions like weather, time of day, and camera viewpoint. Use when the user wants to benchmark on SKYSCENES, UAVid, AEROSCAPES, ICG DRONE, SYNDrone, or asks about evaluating this task. Reports mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2312.06719
  bibtex_key: khose2023skyscenes
  confidence: high
---

# skyscenes-aerial-seg-eval

> SkyScenes: A Synthetic Dataset for Aerial Scene Understanding — Khose et al. (2023) (arXiv:2312.06719, 2023)

## What this evaluates

Evaluates semantic segmentation models trained on synthetic aerial imagery for their ability to generalize to real-world UAV datasets and adapt to varying environmental conditions like weather, time of day, and camera viewpoint.

## Datasets

- **SKYSCENES** — total ?; splits: train (-1)
- **UAVid** — total ?; splits: test (-1)
- **AEROSCAPES** — total ?; splits: test (-1)
- **ICG DRONE** — total ?; splits: test (-1)
- **SYNDrone** — total ?; splits: train (-1)

## Metrics

- `mIoU` **(primary)** — range: [0, 1]
  - Mean Intersection over Union, computed as the average of IoU across all semantic classes. IoU per class is calculated as TP / (TP + FP + FN).
- `Test IoU` — range: [0, 1]
  - Per-class Intersection over Union reported for specific target classes (e.g., vehicle, human) to evaluate tail-class generalization.

## Input / output format

**Input**: RGB aerial images (and optionally depth maps in multi-modal experiments) captured from UAVs at varying altitudes, pitch angles, weather conditions, and times of day.

**Output**: Per-pixel semantic segmentation masks assigning each pixel to a predefined class (e.g., road, building, vehicle, human, tree, low-veg, clutter).

## Scoring recipe

```python
def compute_miou(pred_masks, gt_masks, num_classes):
    ious = []
    for c in range(num_classes):
        pred_c = (pred_masks == c)
        gt_c = (gt_masks == c)
        intersection = np.logical_and(pred_c, gt_c).sum()
        union = np.logical_or(pred_c, gt_c).sum()
        iou = intersection / union if union > 0 else 0.0
        ious.append(iou)
    return np.mean(ious)
```

## Common pitfalls

- Class vocabularies differ between synthetic and real datasets; models require explicit class merging/alignment before evaluation.
- Real-world datasets are captured at different heights and pitch angles; evaluation subsets must be aligned with corresponding synthetic viewpoint conditions.
- Low-shot experiments use specific fixed percentages (5%, 10%, 25%, 50%, 100%) of the real training set, not random splits.

## Evidence (verbatim from paper)

> We assess performance on 3 real-world aerial datasets - UAVid [27], AEROSCAPEs [34], ICG DRONE [20]. Since different datasets have different class vocabularies and definitions, for our experiments, we adapt the class vocabulary of the synthetic source dataset to that of the target real-world datasets (see Sec. C.1 in appendix for class merging and assignment schemes). Table 4: SKYSCENES augmented real data improves performance in low shot regimes. We compare DeepLabv2 [4], DAFormer [16], and Rein [54] models trained using varying percentages of labeled UAVid [27] images. Models are either trained jointly on SKYSCENES and UAVid (JT) or pretrained on SKYSCENES and finetuned on UAVid (FT). Augmenting real data with SKYSCENES enhances real-world generalization in low-shot scenarios. Real World mIoU (↑)

## Citation

```bibtex
@misc{khose2023skyscenes,
  title={SkyScenes: A Synthetic Dataset for Aerial Scene Understanding},
  author={Khose et al. (2023)},
  year={2023},
  note={arXiv:2312.06719}
}
```

- arXiv: 2312.06719