Map Based Fdi Eval

---
name: map-based-fdi-eval
description: This evaluation probes the operational utility of data-driven Fire Danger Index (FDI) models for wildfire forecasting. It assesses both point-level classification accuracy and full-map spatial inference performance, explicitly quantifying detection rates and false positive distributions under realistic deployment conditions. Use when the user wants to benchmark on FireCube, or asks about evaluating this task. Reports Map-based Recall Percentiles.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.25469
  bibtex_key: [REDACTED]
  confidence: high
---

# map-based-fdi-eval

> Not a fragment, but the whole: Map-based evaluation of data-driven Fire Danger Index models — Alvi et al. (2026) (arXiv:2603.25469, 2026)

## What this evaluates

This evaluation probes the operational utility of data-driven Fire Danger Index (FDI) models for wildfire forecasting. It assesses both point-level classification accuracy and full-map spatial inference performance, explicitly quantifying detection rates and false positive distributions under realistic deployment conditions.

## Datasets

- **FireCube** — total ?; splits: test (-1), validation (-1)

## Metrics

- `Recall` — range: [0, 1]
  - Percentage of fires correctly recognized (where predicted FDI > 0.5) out of total actual fires on a given day.
- `Precision` — range: [0, 1]
  - Ratio of correctly predicted fire locations to all locations predicted as fire.
- `F1-score` — range: [0, 1]
  - Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall).
- `Map-based Recall Percentiles` **(primary)** — range: [0, 1]
  - Daily recall values are computed for each day with a fire, then aggregated into Q-th percentiles (e.g., 50% percentile indicates the recall threshold exceeded on 50% of fire days).
- `False Positive Rate (Skewness)` — range: other
  - Assessed by computing the skewness of the FDI value distribution on randomly selected no-fire days; positive skew indicates fewer false positives.

## Input / output format

**Input**: Daily environmental and weather features for fire-susceptible grid cells/locations, filtered by land cover type.

**Output**: A daily Fire Danger Index (FDI) map assigning a continuous risk score to each fire-susceptible location.

## Scoring recipe

```python
def compute_standard_metrics(preds, golds):
    tp = sum(p > 0.5 and g > 0.5 for p, g in zip(preds, golds))
    fp = sum(p > 0.5 and g == 0 for p, g in zip(preds, golds))
    fn = sum(p == 0 and g > 0.5 for p, g in zip(preds, golds))
    recall = tp / (tp + fn)
    precision = tp / (tp + fp)
    f1 = 2 * precision * recall / (precision + recall)
    return recall, precision, f1

def compute_map_recall_percentiles(daily_preds, daily_golds):
    daily_recalls = []
    for day in fire_days:
        day_tp = sum(p > 0.5 and g > 0.5 for p, g in zip(daily_preds[day], daily_golds[day]))
        day_total = sum(g > 0.5 for g in daily_golds[day])
        daily_recalls.append(day_tp / day_total)
    return np.percentile(daily_recalls, [40, 50, 60, 70, 80, 90])
```

## Common pitfalls

- Test set samples are balanced/curated, which may overestimate performance compared to full-map operational inference where class distribution is highly imbalanced.
- FDI values for non-fire-susceptible areas (urban, water) are excluded from evaluation, so models must correctly mask or ignore these regions.
- Map-based recall percentiles aggregate daily performance, potentially masking days with extremely poor detection that are averaged out by high-recall days.

## Evidence (verbatim from paper)

> We compute daily full-map inference for the fire season (JAS) in 2020 and 2021 (149 days in total). Recall - the percentage of fires correctly recognized (having FDI > 0.5) - is computed from each day when there was a fire. The model performance is expressed in the 40%, 50%, 60%, 70%, 80%, and 90% percentiles. The Q-th percentile is the upper limit of recall on Q% of the days.

## Citation

```bibtex
@misc{alvi2026mapbasedfdi,
  title={Not a fragment, but the whole: Map-based evaluation of data-driven Fire Danger Index models},
  author={Alvi et al. (2026)},
  year={2026},
  note={arXiv:2603.25469}
}
```

- arXiv: 2603.25469