Oem Gfss Eval

---
name: oem-gfss-eval
description: Evaluates a model's ability to perform generalized few-shot semantic segmentation on remote sensing imagery. It tests whether a model can accurately segment both previously seen (base) and new (novel) land cover classes simultaneously using only a few support examples (5-shot), probing generalization and resistance to class forgetting in low-data regimes. Use when the user wants to benchmark on OEM-GFSS, or asks about evaluating this task. Reports mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2409.11227
  bibtex_key: broni-bediako2024oem
  confidence: medium
---

# oem-gfss-eval

> Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark — Broni-Bediako et al. (2024) (arXiv:2409.11227, 2024)

## What this evaluates

Evaluates a model's ability to perform generalized few-shot semantic segmentation on remote sensing imagery. It tests whether a model can accurately segment both previously seen (base) and new (novel) land cover classes simultaneously using only a few support examples (5-shot), probing generalization and resistance to class forgetting in low-data regimes.

## Datasets

- **OEM-GFSS** — total 408; splits: train (258), val (50), test (100); repo https://github.com/cliffbb/OEM-Fewshot-Challenge

## Metrics

- `mIoU` **(primary)** — range: [0, 1]
  - Mean Intersection over Union across all 15 classes. Computed as the average of per-class IoU, where IoU = true positives / (true positives + false positives + false negatives).

## Input / output format

**Input**: 1024×1024 pixel remote sensing images (0.25–0.5m resolution). Each evaluation instance provides a support set with pixel-level masks for novel classes and a query set containing images and masks for both base and novel classes.

**Output**: Pixel-wise segmentation mask for each query image, assigning one of 15 fine-grained land cover class labels or background.

## Scoring recipe

```python
def compute_miou(predictions, ground_truth, num_classes=15):
    ious = []
    for c in range(num_classes):
        pred_c = (predictions == c)
        gt_c = (ground_truth == c)
        intersection = np.logical_and(pred_c, gt_c).sum()
        union = np.logical_or(pred_c, gt_c).sum()
        ious.append(intersection / union if union > 0 else 1.0)
    return np.mean(ious)
```

## Common pitfalls

- Evaluating only on novel classes instead of jointly on base and novel classes, which violates the generalized few-shot setting.
- Ignoring the 5-shot constraint for the support set or using more examples than specified.
- Failing to correctly handle the background class, as undefined objects are explicitly labeled as background (RGB 0,0,0) and should be included in the evaluation.

## Evidence (verbatim from paper)

> The validation and test sets contain images and labels of the val-novel and test-novel classes, respectively, and both consist of a support set and a query set for GFSS task of a 5-shot with 4-novel and 7-base classes.

## Citation

```bibtex
@misc{broni-bediako2024oem,
  title={Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark},
  author={Broni-Bediako et al. (2024)},
  year={2024},
  note={arXiv:2409.11227}
}
```

- arXiv: 2409.11227