Tornado Prediction Eval

---
name: tornado-prediction-eval
description: Evaluates machine learning classifiers' ability to predict tornado occurrences up to five days in advance using historical meteorological grid data. It measures detection capability and false alarm rates under a strict temporal train-test split simulating real-world forecasting. Use when the user wants to benchmark on Custom Tornado Forecasting Dataset, or asks about evaluating this task. Reports POD.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2208.05855
  bibtex_key: coccomini2022predictingtornadoes
  confidence: high
---

# tornado-prediction-eval

> Predicting Tornadoes days ahead with Machine Learning — Coccomini and Zara (2022) (arXiv:2208.05855, 2022)

## What this evaluates

Evaluates machine learning classifiers' ability to predict tornado occurrences up to five days in advance using historical meteorological grid data. It measures detection capability and false alarm rates under a strict temporal train-test split simulating real-world forecasting.

## Datasets

- **Custom Tornado Forecasting Dataset** — total 5103; splits: train (4993), test (200)

## Metrics

- `POD` **(primary)** — range: [0, 1]
  - POD = tp / (tp + fn)
- `FAR` — range: [0, 1]
  - FAR = fp / (tp + fp)

## Input / output format

**Input**: Five daily meteorological grids of size 19×19 cells per event instance, containing temperature, wind, precipitation, and cloud metrics.

**Output**: Binary classification label (tornadic vs. non-tornadic) or prediction probability.

## Scoring recipe

```python
def compute_metrics(tp, fp, fn):
    pod = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    far = fp / (tp + fp) if (tp + fp) > 0 else 0.0
    return {'POD': pod, 'FAR': far}
```

## Common pitfalls

- Temporal split is strict: train on 1990–2016, test only on 2017 to simulate real-time forecasting.
- Non-tornadic instances are defined by a 10-day gap rule from any tornadic event, not random negative sampling.
- Input is pre-aggregated 19×19 grids, not raw point measurements or continuous time series.

## Evidence (verbatim from paper)

> Classifiers, evaluated using two quality metrics commonly used in this field, Probability Of Detection,  $POD = \frac{tp}{(tp + fn)}$  and False Alarm Rate,  $FAR = \frac{fp}{(tp + fp)}$  are listed in Table 2.

## Citation

```bibtex
@misc{coccomini2022predictingtornadoes,
  title={Predicting Tornadoes days ahead with Machine Learning},
  author={Coccomini and Zara (2022)},
  year={2022},
  note={arXiv:2208.05855}
}
```

- arXiv: 2208.05855