Flair One Eval

---
name: flair-one-eval
description: Evaluates the ability of models to perform high-resolution land-cover semantic segmentation on aerial imagery. It probes robustness to spatial, temporal, and multi-sensor domain shifts, as well as handling radiometric inconsistencies and phenological variations across diverse landscapes. Use when the user wants to benchmark on FLAIR-one, or asks about evaluating this task. Reports mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2211.12979
  bibtex_key: garioud2022flair
  confidence: high
---

# flair-one-eval

> FLAIR #1: semantic segmentation and domain adaptation dataset — Garioud et al. (2022) (arXiv:2211.12979, 2022)

## What this evaluates

Evaluates the ability of models to perform high-resolution land-cover semantic segmentation on aerial imagery. It probes robustness to spatial, temporal, and multi-sensor domain shifts, as well as handling radiometric inconsistencies and phenological variations across diverse landscapes.

## Datasets

- **FLAIR-one** — total 77412; splits: train (61712), test (15700); repo https://github.com/IGNF/odeon-landcover

## Metrics

- `mIoU` **(primary)** — range: [0, 1]
  - Mean Intersection over Union, computed as the average of the Intersection over Union (IoU) across all 13 semantic classes. IoU for a class is the ratio of correctly predicted pixels to the union of predicted and ground truth pixels.

## Input / output format

**Input**: High-resolution (0.2m) aerial image patches, optionally accompanied by acquisition metadata.

**Output**: Per-pixel semantic segmentation mask assigning one of 13 land-cover classes.

## Scoring recipe

```python
def compute_miou(preds, targets, num_classes=13):
    ious = []
    for c in range(num_classes):
        tp = ((preds == c) & (targets == c)).sum()
        fp = ((preds == c) & (targets != c)).sum()
        fn = ((preds != c) & (targets == c)).sum()
        ious.append(tp / (tp + fp + fn + 1e-6))
    return sum(ious) / len(ious)
```

## Common pitfalls

- Models struggle with low-frequency classes (e.g., bare soil, coniferous), achieving IoU < 0.4.
- High inter-class confusion occurs between semantically similar categories like herbaceous vegetation and agricultural land.
- Metadata integration and standard geometric augmentations did not yield performance gains in the reported baseline, suggesting careful modality fusion or class-aware augmentation is needed.

## Evidence (verbatim from paper)

> The results obtained using 61,712 patches for training, and testing on the remaining 15,700 patches of the FLAIR-one dataset are reported in Table II. The given results are average and their standard deviation of $5\mathrm{mIoU}$ scores obtained for 5 runs in a given configuration. Detailed per-class IoU results for the baseline (without metadata integration or data augmentations) are illustrated in Figure 9.

## Citation

```bibtex
@misc{garioud2022flair,
  title={FLAIR #1: semantic segmentation and domain adaptation dataset},
  author={Garioud et al. (2022)},
  year={2022},
  note={arXiv:2211.12979}
}
```

- arXiv: 2211.12979