Rf Climate Param Eval

---
name: rf-climate-param-eval
description: Evaluates whether a random forest model can accurately emulate subgrid atmospheric processes (vertical advection, cloud microphysics, turbulent diffusion, surface fluxes, radiative heating) from high-resolution simulation data. It further tests if the learned parameterization enables stable, long-term coarse-resolution climate simulations that reproduce key statistics like mean and extreme precipitation and ITCZ structure compared to high-resolution ground truth. Use when the user wants to benchmark on SAM aquaplanet simulation (high-resolution output), or asks about evaluating this task. Reports R^2.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2001.03151
  bibtex_key: yuval2020stable
  confidence: high
---

# rf-climate-param-eval

> Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions — Yuval et al. (2020) (arXiv:2001.03151, 2020)

## What this evaluates

Evaluates whether a random forest model can accurately emulate subgrid atmospheric processes (vertical advection, cloud microphysics, turbulent diffusion, surface fluxes, radiative heating) from high-resolution simulation data. It further tests if the learned parameterization enables stable, long-term coarse-resolution climate simulations that reproduce key statistics like mean and extreme precipitation and ITCZ structure compared to high-resolution ground truth.

## Datasets

- **SAM aquaplanet simulation (high-resolution output)** — total ?; splits: train (-1), test (-1)

## Metrics

- `R^2` **(primary)** — range: [0, 1]
  - Coefficient of determination: R^2 = 1 - (SS_res / SS_tot), where SS_res is the sum of squared residuals between predicted and true subgrid tendencies, and SS_tot is the total sum of squares relative to the mean of the true values. Measures the proportion of variance in the target explained by the model.
- `RMSE` — range: other
  - Root mean square error: sqrt(mean((y_true - y_pred)^2)). Measures the standard deviation of prediction errors in the same physical units as the target variable (e.g., kg kg^-1 s^-1 for tendencies, mm/day for precipitation).

## Input / output format

**Input**: Vertical profiles of resolved temperature (T), total non-precipitating water mixing ratio (q_T), precipitating water mixing ratio (q_p), and distance from equator (|y|) for RF-tend; plus lower-tropospheric profiles of T, q_T, zonal wind (u), meridional wind (v), surface wind speed, and |y| for RF-diff.

**Output**: RF-tend: 144 outputs (48 model levels × 3 variables: subgrid tendencies for h_L, q_T, q_p). RF-diff: 17 outputs (15 model levels of turbulent diffusivity D, plus subgrid surface fluxes for h_L and q_T).

## Scoring recipe

```python
import numpy as np
def compute_metrics(y_true, y_pred):
    y_true = np.asarray(y_true)
    y_pred = np.asarray(y_pred)
    ss_res = np.sum((y_true - y_pred) ** 2)
    ss_tot = np.sum((y_true - np.mean(y_true)) ** 2)
    r2 = 1.0 - (ss_res / ss_tot) if ss_tot != 0 else 0.0
    rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
    return {'R2': r2, 'RMSE': rmse}
```

## Common pitfalls

- Confusing offline performance (R^2/RMSE on withheld data) with online performance (stability and climate statistics in long-term coarse-resolution simulations), as they vary non-monotonically with grid spacing.
- Comparing R^2 across different grid spacings without accounting for the stochastic component that averages out during coarse-graining, which can make larger grid spacings appear better offline despite worse online stability.
- Not coarse-graining the high-resolution ground truth to the same grid spacing as the coarse simulation before calculating extreme precipitation percentiles, leading to unfair comparisons.

## Evidence (verbatim from paper)

> The offline performance as measured by the coefficient of determination (R^2) improves substantially as the grid spacing increases (Fig. 3a, compare Fig. 3c and 3e), consistent with the idea of more predictable subgrid tendencies with more averaging over larger grid boxes.

## Citation

```bibtex
@misc{yuval2020stable,
  title={Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions},
  author={Yuval et al. (2020)},
  year={2020},
  note={arXiv:2001.03151}
}
```

- arXiv: 2001.03151