Lewmm Physical Eval

---
name: lewmm-physical-eval
description: Evaluates whether a latent world model captures physical structure and dynamics by probing latent representations for physical quantities and measuring predictive surprise under physical versus visual perturbations. Use when the user wants to benchmark on TwoRoom, PushT, OGBench-Cube, Reacher, or asks about evaluating this task. Reports MSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.19312
  bibtex_key: maes2026lewmm
  confidence: high
---

# lewmm-physical-eval

> LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels — Maes et al. (2026) (arXiv:2603.19312, 2026)

## What this evaluates

Evaluates whether a latent world model captures physical structure and dynamics by probing latent representations for physical quantities and measuring predictive surprise under physical versus visual perturbations.

## Datasets

- **TwoRoom, PushT, OGBench-Cube, Reacher** — total ?; splits: test (-1)

## Metrics

- `MSE` **(primary)** — range: other
  - Mean squared error between predicted and ground-truth physical quantities (e.g., position, velocity). Lower is better.
- `Pearson correlation (r)` — range: other
  - Pearson correlation coefficient between predicted and ground-truth quantities. Higher is better.
- `Surprise` — range: other
  - Predictive loss or negative log-likelihood computed on perturbed trajectories relative to unperturbed baselines to measure violation of physical expectations.

## Input / output format

**Input**: Latent embeddings extracted from context frames (for probing) or raw pixel trajectories with visual/physical perturbations (for VoE).

**Output**: Predicted physical quantities (e.g., 2D position, joint velocity, quaternion) or scalar surprise scores per timestep.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth):
    mse = np.mean((predictions - ground_truth) ** 2)
    r = pearsonr(predictions, ground_truth).statistic
    return mse, r
def compute_surprise(perturbed_loss, unperturbed_loss):
    return perturbed_loss - unperturbed_loss
```

## Common pitfalls

- Probing metrics are reported separately for linear and non-linear (MLP) probes; results differ significantly and must not be conflated.
- Surprise signals are averaged over multiple perturbation types; physical vs. visual perturbations must be distinguished to assess physical understanding.
- Control evaluation uses fixed budgets and goal distances per environment; mixing these settings invalidates comparisons.

## Evidence (verbatim from paper)

> For each probe, we report the mean squared error (MSE) and the Pearson correlation coefficient between the predicted and ground-truth quantities.

## Citation

```bibtex
@misc{maes2026lewmm,
  title={LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels},
  author={Maes et al. (2026)},
  year={2026},
  note={arXiv:2603.19312}
}
```

- arXiv: 2603.19312