Fgn Weather Forecasting Eval

---
name: fgn-weather-forecasting-eval
description: Evaluates a functional generative network for medium-range probabilistic weather forecasting against operational ground truth (HRES-fc0) and a diffusion-based baseline (GenCast). Probes the model's ability to capture joint spatial structures and predict tropical cyclone tracks using deterministic and probabilistic scoring rules. Use when the user wants to benchmark on HRES-fc0, ERA5, or asks about evaluating this task. Reports probabilistic metrics.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.10772
  bibtex_key: alet2025skillful
  confidence: medium
---

# fgn-weather-forecasting-eval

> Skillful joint probabilistic weather forecasting from marginals — Alet et al. (2025) (arXiv:2506.10772, 2025)

## What this evaluates

Evaluates a functional generative network for medium-range probabilistic weather forecasting against operational ground truth (HRES-fc0) and a diffusion-based baseline (GenCast). Probes the model's ability to capture joint spatial structures and predict tropical cyclone tracks using deterministic and probabilistic scoring rules.

## Datasets

- **HRES-fc0** — total ?; splits: val (-1), test (-1)
- **ERA5** — total ?; splits: pre-training (-1)

## Metrics

- `probabilistic metrics` **(primary)** — range: other
  - Not specified in provided section; typically encompasses ensemble scoring rules such as CRPS, log-likelihood, or reliability diagrams for probabilistic forecasts.

## Input / output format

**Input**: Gridded atmospheric state variables at 0.25° resolution from HRES-fc0 or ERA5, used as conditioning inputs for the generative model.

**Output**: Ensemble of 15-day forecast trajectories with 6-hour timesteps, representing predicted atmospheric fields.

## Scoring recipe

```python
def compute_metric(predictions, gold):
    # Exact formula not detailed in provided section
    # predictions: list of ensemble forecast trajectories
    # gold: ground truth HRES-fc0 analysis fields
    score = aggregate_probabilistic_scores(predictions, gold)
    return score
```

## Common pitfalls

- Specific metric definitions and scoring thresholds are not detailed in this section; refer to the full paper for exact formulas.
- Evaluation uses strict temporal splits (2022 validation, 2023 test) rather than random shuffling, making temporal leakage a critical concern.

## Evidence (verbatim from paper)

> achieving state-of-the-art performance in deterministic and probabilistic metrics, superior joint spatial structure, and significantly improved tropical cyclone track predictions compared to GenCast and ENS.

## Citation

```bibtex
@misc{alet2025skillful,
  title={Skillful joint probabilistic weather forecasting from marginals},
  author={Alet et al. (2025)},
  year={2025},
  note={arXiv:2506.10772}
}
```

- arXiv: 2506.10772