Realpdebench Eval

---
name: realpdebench-eval
description: Evaluates scientific machine learning models on sim-to-real transfer for complex physical systems. It probes a model's ability to predict spatiotemporal dynamics from real-world measurements, leveraging simulated pretraining, and assesses long-term prediction stability under autoregressive rollout. Use when the user wants to benchmark on Cylinder, ControlledCylinder, FSI, Foil, Combustion, or asks about evaluating this task. Reports RMSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.01829
  bibtex_key: hu2026realpdebench
  confidence: high
---

# realpdebench-eval

> RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data — Hu et al. (2026) (arXiv:2601.01829, 2026)

## What this evaluates

Evaluates scientific machine learning models on sim-to-real transfer for complex physical systems. It probes a model's ability to predict spatiotemporal dynamics from real-world measurements, leveraging simulated pretraining, and assesses long-term prediction stability under autoregressive rollout.

## Datasets

- **Cylinder** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/AI4Science-WestlakeU/RealPDEBench
- **ControlledCylinder** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/AI4Science-WestlakeU/RealPDEBench
- **FSI** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/AI4Science-WestlakeU/RealPDEBench
- **Foil** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/AI4Science-WestlakeU/RealPDEBench
- **Combustion** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/AI4Science-WestlakeU/RealPDEBench

## Metrics

- `RMSE` **(primary)** — range: other
  - Root Mean Squared Error between predicted and ground truth spatiotemporal fields.
- `Rel L2` — range: [0, 1]
  - Relative L2 Error, computed as the L2 norm of the prediction error divided by the L2 norm of the ground truth.
- `fRMSE` — range: other
  - Frequency RMSE, measuring prediction error in the frequency domain to capture global periodicity and dynamics.
- `Update Ratio` — range: [0, 1]
  - Ratio of update iterations required for real-world finetuning to reach optimal performance compared to direct real-world training.

## Input / output format

**Input**: Spatiotemporal PDE data consisting of input and output fields over T time steps. Simulated inputs may contain unmeasured modalities that are randomly masked during training.

**Output**: Predicted spatiotemporal fields over T time steps (or NT steps for autoregressive evaluation).

## Scoring recipe

```python
def compute_metrics(pred, gold):
    rmse = np.sqrt(np.mean((pred - gold) ** 2))
    rel_l2 = np.linalg.norm(pred - gold) / np.linalg.norm(gold)
    f_pred = np.fft.fft(pred)
    f_gold = np.fft.fft(gold)
    f_rmse = np.sqrt(np.mean(np.abs(f_pred - f_gold) ** 2))
    return rmse, rel_l2, f_rmse
```

## Common pitfalls

- Data splits are performed at the parameter level, not trajectory level, meaning different physical configurations are held out for testing.
- Autoregressive evaluation compounds error rapidly; single-round and multi-round (e.g., 10-round) results can differ drastically, requiring careful reporting of rollout steps.
- Simulated data contains extra modalities not present in real-world measurements; models must be trained with random masking of these unmeasured modalities to avoid unfair advantages.

## Evidence (verbatim from paper)

> We report the RMSE, Relative L2 Error, fRMSE, and Update Ratio of all datasets and baselines under the three training categories, as summarized in Table 1.

## Citation

```bibtex
@misc{hu2026realpdebench,
  title={RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data},
  author={Hu et al. (2026)},
  year={2026},
  note={arXiv:2601.01829}
}
```

- arXiv: 2601.01829