Bounded Nbeddyn Eval

---
name: bounded-nbeddyn-eval
description: This evaluation probes a model's ability to forecast and reconstruct the dynamics of partially observed, chaotic geophysical systems. It specifically tests short-term prediction accuracy and long-term topological stability (boundedness) under both in-distribution and out-of-attractor initial conditions. Use when the user wants to benchmark on Lorenz-63, Lorenz-96, or asks about evaluating this task. Reports RMSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2202.05750
  bibtex_key: ouala2022bounded
  confidence: high
---

# bounded-nbeddyn-eval

> Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning — Said Ouala et al. (arXiv:2202.05750, 2022)

## What this evaluates

This evaluation probes a model's ability to forecast and reconstruct the dynamics of partially observed, chaotic geophysical systems. It specifically tests short-term prediction accuracy and long-term topological stability (boundedness) under both in-distribution and out-of-attractor initial conditions.

## Datasets

- **Lorenz-63** — total 5000; splits: train (4000), test (1000); repo https://github.com/CIA-Oceanix/Bounded-NbedDyn
- **Lorenz-96** — total 5000; splits: train (4000), test (1000); repo https://github.com/CIA-Oceanix/Bounded-NbedDyn

## Metrics

- `RMSE` **(primary)** — range: other
  - Root mean square error between predicted and ground-truth state variables over the test set.
- `Largest Lyapunov Exponent` — range: other
  - Computed from a long-term simulation (10,000 time steps) of the predicted trajectory to measure chaotic divergence and topological fidelity.

## Input / output format

**Input**: Sequential partial observations of the system (e.g., first variable for Lorenz-63, first 20 states for Lorenz-96) sampled at dt=0.01. Benchmarks also use delay-embedding representations of these observations.

**Output**: Forecasted state sequences for the full system (or observed subset) over future time steps, evaluated at t0+dt and t0+4dt for short-term, and up to 10,000 steps for long-term stability.

## Scoring recipe

```python
def compute_rmse(pred, true):
    return np.sqrt(np.mean((pred - true) ** 2))

def compute_lyapunov(traj, dt=0.01, steps=10000):
    # Compute largest Lyapunov exponent using standard algorithm (e.g., Rosenstein)
    # on the first `steps` time steps of the predicted trajectory
    return lyapunov_exponent(traj[:steps], dt)
```

## Common pitfalls

- Unconstrained models often diverge to infinity when tested with initial conditions outside the training attractor, making long-term metrics undefined.
- Benchmark baselines (SR, RNN, Latent-ODE) are highly sensitive to delay-embedding hyperparameters (lag tau, dimension d_E), which must be carefully tuned via mutual information or false nearest neighbors.
- The Lyapunov exponent requires a sufficiently long simulation (10,000 steps) and is only meaningful if the trajectory remains bounded.

## Evidence (verbatim from paper)

> We evaluate in Table 1 the short-term forecast as well as the topological structure of the simulated limit-sets (illustrated for instance through the largest Lyapunov exponent). Regarding the short-term forecast, both versions of the NbedDyn model outperform benchmarked state-of-the-art techniques in terms of root mean square error (RMSE).

## Citation

```bibtex
@misc{ouala2022bounded,
  title={Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning},
  author={Said Ouala et al.},
  year={2022},
  note={arXiv:2202.05750}
}
```

- arXiv: 2202.05750