Wind Power Ensemble Eval

---
name: wind-power-ensemble-eval
description: Evaluates the calibration, sharpness, and accuracy of probabilistic wind power forecasts under different ensemble post-processing strategies (raw, weather-only, power-only, and joint weather-power post-processing). It probes whether correcting biases at the weather stage alone is sufficient, or if direct post-processing of the final power ensemble is required to handle non-linear power curve biases. Use when the user wants to benchmark on Benchmark Data, Swedish Data Set, or asks about evaluating this task. Reports CRPS.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2009.14127
  bibtex_key: phipps2020evaluating
  confidence: high
---

# wind-power-ensemble-eval

> Evaluating Ensemble Post-Processing for Wind Power Forecasts — Phipps et al. (2020) (arXiv:2009.14127, 2020)

## What this evaluates

Evaluates the calibration, sharpness, and accuracy of probabilistic wind power forecasts under different ensemble post-processing strategies (raw, weather-only, power-only, and joint weather-power post-processing). It probes whether correcting biases at the weather stage alone is sufficient, or if direct post-processing of the final power ensemble is required to handle non-linear power curve biases.

## Datasets

- **Benchmark Data** — total ?; splits: train (-1), test (-1); repo https://github.com/KIT-IAI/EvaluatingEnsemblePostProcessing
- **Swedish Data Set** — total ?; splits: train (-1), test (-1); repo https://github.com/KIT-IAI/EvaluatingEnsemblePostProcessing

## Metrics

- `CRPS` **(primary)** — range: other (MW)
  - Continuous Ranked Probability Score; measures the integrated squared difference between the predictive cumulative distribution function and the Heaviside step function at the observed value. Lower values indicate better accuracy.
- `CRPSS` — range: percent
  - CRPS Skill Score; calculated as 1 - (CRPS_postprocessed / CRPS_raw), where CRPS_raw is the baseline raw ensemble score. Positive values indicate improvement over the raw ensemble.
- `PIT histogram` — range: [0, 1]
  - Probability Integral Transform histogram; plots the cumulative probability of the observed value under the predictive distribution. A well-calibrated forecast yields a uniform histogram on [0, 1].

## Input / output format

**Input**: Ensemble weather forecasts (e.g., TIGGE or ECMWF EPS variables like wind speed, temperature, pressure) and historical wind power generation data.

**Output**: Probabilistic wind power forecasts represented as predictive distributions (e.g., truncated normal) for specified forecast horizons (3h to 24h).

## Scoring recipe

```python
def compute_crps(forecast_cdf, observation):
    return integral((forecast_cdf(x) - (x >= observation))**2 dx)

def compute_crpss(crps_postprocessed, crps_raw):
    return 1.0 - (crps_postprocessed / crps_raw)

def compute_pit_histogram(forecast_cdf, observations):
    pits = [forecast_cdf(obs) for obs in observations]
    return histogram(pits, bins=10)
```

## Common pitfalls

- Post-processing weather ensembles alone does not consistently improve wind power forecast calibration because the non-linear wind power curve introduces additional biases that require direct power-level correction.
- CRPSS is calculated relative to the raw ensemble baseline, not a deterministic or climatology baseline, so negative values indicate worse performance than raw ensembles.
- The benchmark dataset uses synthetically generated wind power data, which may not capture real-world structural changes like capacity upgrades or maintenance outages present in the Swedish dataset.

## Evidence (verbatim from paper)

> We first perform the different post-processing strategies on the benchmark data set and evaluate the calibration and sharpness based on the above introduced CRPS and PIT/ verification rank histogram for both forecasting models with varying forecast horizon.

## Citation

```bibtex
@misc{phipps2020evaluating,
  title={Evaluating Ensemble Post-Processing for Wind Power Forecasts},
  author={Phipps et al. (2020)},
  year={2020},
  note={arXiv:2009.14127}
}
```

- arXiv: 2009.14127