Noaa Sst Forecasting Eval

---
name: noaa-sst-forecasting-eval
description: Evaluates the ability of data-driven models to forecast low-dimensional geophysical dynamics (sea surface temperature and air temperature) from historical time-series observations. It probes long-horizon prediction accuracy, bias-variance trade-offs, and computational efficiency compared to physics-based and deep learning baselines. Use when the user wants to benchmark on NOAA-SST, NOAA-NCEP NAM, or asks about evaluating this task. Reports RMSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2103.10935
  bibtex_key: hamzi2021data
  confidence: high
---

# noaa-sst-forecasting-eval

> Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods — Hamzi et al. (2021) (arXiv:2103.10935, 2021)

## What this evaluates

Evaluates the ability of data-driven models to forecast low-dimensional geophysical dynamics (sea surface temperature and air temperature) from historical time-series observations. It probes long-horizon prediction accuracy, bias-variance trade-offs, and computational efficiency compared to physics-based and deep learning baselines.

## Datasets

- **NOAA-SST** — total 1914; splits: train (427), test (1487)
- **NOAA-NCEP NAM** — total ?; splits: train (-1), test (-1)

## Metrics

- `RMSE` **(primary)** — range: other
  - Root Mean Squared Error between predicted and observed values, measured in degrees Celsius. Calculated as the square root of the mean of squared differences over the testing period.
- `Correlation coefficient` — range: [-1, 1]
  - Pearson correlation coefficient measuring linear relationship between forecast and truth across the spatial domain.
- `Cosine similarity` — range: [0, 1]
  - Cosine similarity between forecast and truth vectors, used to assess the ability to detect extreme fluctuations.

## Input / output format

**Input**: Time-delayed input windows of historical geophysical observations (e.g., 7-day sequences) projected onto a low-dimensional Proper Orthogonal Decomposition (POD) basis or used directly.

**Output**: Forecasted values for a specified future horizon (e.g., 8 weeks or 7 days ahead), either as POD coefficients or direct temperature/field values.

## Scoring recipe

```python
def compute_rmse(pred, true):
    return np.sqrt(np.mean((pred - true) ** 2))

def compute_correlation(pred, true):
    return np.corrcoef(pred.flatten(), true.flatten())[0, 1]

def compute_cosine_similarity(pred, true):
    dot = np.dot(pred.flatten(), true.flatten())
    norm = np.linalg.norm(pred) * np.linalg.norm(true)
    return dot / norm if norm > 0 else 0.0
```

## Common pitfalls

- The evaluation relies on time-delayed input windows rather than single-step predictions, requiring careful alignment of temporal strides between input sequences and target horizons.
- Computational cost baselines (e.g., LSTM) often exclude neural architecture search overhead, making direct wall-time comparisons unfair unless search costs are explicitly accounted for.
- Error characteristics differ significantly between datasets: NOAA-SST exhibits bias-dominated errors (low noise), while NOAA-NCEP NAM shows variance-dominated errors (high noise), which can mislead model selection if not analyzed separately.

## Evidence (verbatim from paper)

> The reconstruction accuracy from the forecast is compared in a series of assessments beginning with root-mean-squared error (RMSE) assessments as shown in Figure 8 for the testing time period. POD-RKHS is seen to provide competitive results in comparison to persistence and climatology at the lower latitudes.

## Citation

```bibtex
@misc{hamzi2021data,
  title={Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods},
  author={Hamzi et al. (2021)},
  year={2021},
  note={arXiv:2103.10935}
}
```

- arXiv: 2103.10935