Cabuar Burned Area Delineation Eval

---
name: cabuar-burned-area-delineation-eval
description: This benchmark evaluates a model's ability to accurately delineate wildfire-affected regions from satellite imagery. It probes pixel-level binary segmentation and change detection capabilities using pre- and post-fire Sentinel-2 multispectral data. Use when the user wants to benchmark on CaBuAr, or asks about evaluating this task. Reports pixel-level accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2401.11519
  bibtex_key: regecambrin2024cabuar
  confidence: medium
---

# cabuar-burned-area-delineation-eval

> CaBuAr: California Burned Areas dataset for delineation — Rege Cambrin et al. (2024) (arXiv:2401.11519, 2024)

## What this evaluates

This benchmark evaluates a model's ability to accurately delineate wildfire-affected regions from satellite imagery. It probes pixel-level binary segmentation and change detection capabilities using pre- and post-fire Sentinel-2 multispectral data.

## Datasets

- **CaBuAr** — total ?; splits: (unstated); repo https://github.com/DarthReca/CaBuAr

## Metrics

- `pixel-level accuracy` **(primary)** — range: [0, 1]
  - Standard binary classification metric calculated as the ratio of correctly classified pixels (burned and undamaged) to the total number of pixels.
- `separability index (SI)` — range: other
  - Ratio of between-class variance to within-class variance for a given spectral index, measuring how well the index separates burned from unburned pixels.

## Input / output format

**Input**: Multispectral Sentinel-2 L2A raster images (pre-fire, post-fire, or both).

**Output**: Binary image mask where each pixel is labeled as either burned or undamaged.

## Scoring recipe

```python
def evaluate_segmentation(pred_mask, gt_mask):
    correct = (pred_mask == gt_mask).sum()
    total = gt_mask.size
    accuracy = correct / total
    return {'accuracy': accuracy}

# For spectral index assessment (Task iv):
# SI = between_class_variance / within_class_variance
# Compute per index using ground truth labels as class indicators.
```

## Common pitfalls

- Models may overfit to specific fire events or geographic regions without cross-validation across different years.
- Spectral index thresholds are highly sensitive to phenological conditions and atmospheric corrections, requiring consistent preprocessing.
- Confusing pre-fire and post-fire inputs can lead to incorrect change detection or segmentation results.

## Evidence (verbatim from paper)

> The proposed dataset can be used as a benchmark for different tasks in supervised and unsupervised scenarios: (i) binary segmentation based on post-fire acquisitions only, (ii) binary segmentation based on pre-fire and post-fire acquisitions, (iii) binary change detection on pre-fire and post-fire images, and (iv) performance assessment of spectral indexes. ... researchers may use the dataset to evaluate the quality of the index by computing the separability index (SI).

## Citation

```bibtex
@misc{regecambrin2024cabuar,
  title={CaBuAr: California Burned Areas dataset for delineation},
  author={Rege Cambrin et al. (2024)},
  year={2024},
  note={arXiv:2401.11519}
}
```

- arXiv: 2401.11519