Model Recovery Eval

---
name: model-recovery-eval
description: This benchmark evaluates the accuracy and hardware efficiency of neural flow-based architectures for recovering underlying dynamics from time-series data. It probes the model's ability to estimate parameters of nonlinear dynamical systems while measuring computational resource constraints like runtime, power, and memory footprint on edge hardware. Use when the user wants to benchmark on Chaotic Lorenz, F8 Cruiser, Lotka Volterra, Pathogenic Attack System, Automated Insulin Delivery (OhioT1D), or asks about evaluating this task. Reports reconstruction MSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.02283
  bibtex_key: xu2025modelrecovery
  confidence: high
---

# model-recovery-eval

> Model Recovery at the Edge under Resource Constraints for Physical AI — Xu et al. (2025) (arXiv:2512.02283, 2025)

## What this evaluates

This benchmark evaluates the accuracy and hardware efficiency of neural flow-based architectures for recovering underlying dynamics from time-series data. It probes the model's ability to estimate parameters of nonlinear dynamical systems while measuring computational resource constraints like runtime, power, and memory footprint on edge hardware.

## Datasets

- **Chaotic Lorenz** — total ?; splits: test (-1)
- **F8 Cruiser** — total ?; splits: test (-1)
- **Lotka Volterra** — total ?; splits: test (-1)
- **Pathogenic Attack System** — total ?; splits: test (-1)
- **Automated Insulin Delivery (OhioT1D)** — total 200; splits: test (200)

## Metrics

- `reconstruction MSE` **(primary)** — range: [0, inf)
  - Mean Square Error between the estimated parameters and the ground truth values.
- `runtime` — range: seconds
  - Execution time measured in seconds using the time library.
- `energy` — range: Joules
  - Total energy consumption in Joules, calculated as average power multiplied by runtime.
- `DRAM footprint` — range: MB
  - Memory usage in megabytes measured using the psutil library.

## Input / output format

**Input**: Time-series sequences of system states (e.g., glucose levels, population counts, chaotic variables) sampled at fixed intervals.

**Output**: Estimated parameters defining the underlying nonlinear dynamical equations, plus hardware metrics (runtime, power, DRAM, energy) for the deployed accelerator.

## Scoring recipe

```python
def compute_mse(est_params, true_params):
    return np.mean((est_params - true_params) ** 2)

def compute_energy(avg_power_w, runtime_s):
    return avg_power_w * runtime_s
```

## Common pitfalls

- Confusing parameter estimation error (reconstruction MSE) with state-trajectory prediction error.
- Assuming energy and memory are always positively correlated; the paper explicitly demonstrates an inverse tradeoff for MR systems.
- Treating hardware metrics (power, DRAM) as algorithm-intrinsic rather than platform-dependent (FPGA vs. GPU).

## Evidence (verbatim from paper)

> Compared to SOTA MR methods such as EMILY and PINN+SR, Table 3 shows that MERINDA achieves comparable or even lower errors across four benchmark applications. Accuracy is measured using Mean Square Error between the estimated parameters and the ground truth values.

## Citation

```bibtex
@misc{xu2025modelrecovery,
  title={Model Recovery at the Edge under Resource Constraints for Physical AI},
  author={Xu et al. (2025)},
  year={2025},
  note={arXiv:2512.02283}
}
```

- arXiv: 2512.02283