Dm Control Prediction Eval

---
name: dm-control-prediction-eval
description: Evaluates long-horizon physical state prediction accuracy of a neural motion simulator in continuous control environments, and measures its effectiveness for zero-shot reinforcement learning by comparing prediction horizons and minimal training step requirements. Use when the user wants to benchmark on DM Control, or asks about evaluating this task. Reports MSE loss.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.07095
  bibtex_key: hao2025neuralmotionsimulator
  confidence: medium
---

# dm-control-prediction-eval

> Neural Motion Simulator: Pushing the Limit of World Models in Reinforcement Learning — Hao et al. (2025) (arXiv:2504.07095, 2025)

## What this evaluates

Evaluates long-horizon physical state prediction accuracy of a neural motion simulator in continuous control environments, and measures its effectiveness for zero-shot reinforcement learning by comparing prediction horizons and minimal training step requirements.

## Datasets

- **DM Control** — total ?; splits: test (-1)

## Metrics

- `MSE loss` **(primary)** — range: other
  - Mean Squared Error between predicted and ground-truth future states over a specified prediction horizon.
- `prediction horizon` — range: other
  - The number of prediction steps at which MoSim achieves the same MSE loss as DreamerV3 at a fixed 16-step horizon.

## Input / output format

**Input**: Current environment state and action sequence in DM Control physics simulation.

**Output**: Predicted future state vectors over a multi-step horizon.

## Scoring recipe

```python
def compute_mse(pred, gold):
    return np.mean((pred - gold) ** 2)

def evaluate_horizon(model, env, baseline_mse):
    for t in range(1, 100):
        pred = model.predict(env.state, env.actions, steps=t)
        gold = env.get_ground_truth(t)
        if compute_mse(pred, gold) <= baseline_mse:
            return t
    return -1
```

## Common pitfalls

- Confusing control steps (physics timesteps) with action repeats; evaluation must account for environment-specific control step multipliers.
- Training TD-MPC2 on random data produces a meaningless latent space; only provided checkpoints should be used for fair comparison.
- Prediction horizon is relative to a fixed 16-step DreamerV3 baseline, not an absolute error threshold.

## Evidence (verbatim from paper)

> Table IV uses the MSE loss of DreamerV3 at a 16-step prediction horizon as a reference. It presents the prediction horizon at which MoSim achieves the same loss, providing a more intuitive measure of MoSim's predictive performance.

## Citation

```bibtex
@misc{hao2025neuralmotionsimulator,
  title={Neural Motion Simulator: Pushing the Limit of World Models in Reinforcement Learning},
  author={Hao et al. (2025)},
  year={2025},
  note={arXiv:2504.07095}
}
```

- arXiv: 2504.07095