Dmcontrol Metaworld Eval

---
name: dmcontrol-metaworld-eval
description: Evaluates sample efficiency, asymptotic performance, and generalization of reinforcement learning agents on high-dimensional continuous control tasks with varying observation modalities (state, pixels, multi-modal) and reward structures (dense, sparse, goal-conditioned). Use when the user wants to benchmark on DeepMind Control Suite (DMControl), Meta-World v2, or asks about evaluating this task. Reports Cumulative Episode Return.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2203.04955
  bibtex_key: hansen2022tdmpc
  confidence: high
---

# dmcontrol-metaworld-eval

> Temporal Difference Learning for Model Predictive Control — Hansen et al. (2022) (arXiv:2203.04955, 2022)

## What this evaluates

Evaluates sample efficiency, asymptotic performance, and generalization of reinforcement learning agents on high-dimensional continuous control tasks with varying observation modalities (state, pixels, multi-modal) and reward structures (dense, sparse, goal-conditioned).

## Datasets

- **DeepMind Control Suite (DMControl)** — total ?; splits: eval (-1)
- **Meta-World v2** — total ?; splits: eval (-1)

## Metrics

- `Cumulative Episode Return` **(primary)** — range: other (task-dependent)
  - Sum of discounted rewards collected over a single episode until termination or a fixed step limit. Reported as mean ± standard deviation across multiple independent random seeds.

## Input / output format

**Input**: Continuous state vectors, or 3 stacked 84x84 RGB frames (with ±4 pixel shift augmentation) for image-based tasks, or multi-modal inputs (proprioception + egocentric camera). Goal-conditioned tasks include target goal states.

**Output**: Continuous action vectors (dimensionality varies by task, e.g., R^6 to R^38).

## Scoring recipe

```python
def evaluate(policy, envs, num_runs=10, steps=100000):
    returns = []
    for _ in range(num_runs):
        ep_returns = []
        for env in envs:
            obs = env.reset()
            ret = 0.0
            for _ in range(steps):
                action = policy(obs)
                obs, reward, done, _ = env.step(action)
                ret += reward
                if done: break
            ep_returns.append(ret)
        returns.append(ep_returns)
    return np.mean(returns, axis=0), np.std(returns, axis=0)
```

## Common pitfalls

- Action repeat of 2 is applied to image-based tasks, meaning environment steps are counted in multiples of 2.
- Planning hyperparameters (horizon and iterations) are task-specific (e.g., 8 iterations for Dog, 12 for Humanoid) rather than fixed globally.
- Some baselines (MuZero, EfficientZero) discretize action spaces, making them infeasible for high-dimensional continuous tasks like Dog, while TD-MPC handles continuous actions natively.

## Evidence (verbatim from paper)

> We evaluate TD-MPC with a TOLD model on a total of 92 diverse and challenging continuous control tasks from DeepMind Control Suite (DMControl; Tassa et al. (2018)) and Meta-World v2 (Yu et al., 2019), including tasks with sparse rewards, high-dimensional state and action spaces, image observations, multi-modal inputs, goal-conditioning, and multi-task learning settings; see Appendix L for task visualizations. We choose these two benchmarks for their great task diversity and availability of baseline implementations and results. Table 1: Learning from pixels. Return of our method (TD-MPC) and state-of-the-art algorithms on the image-based DMControl 100k benchmark used in Srinivas et al. (2020); Kostrikov et al. (2020); Ye et al. (2021). Baselines are tuned specifically for image-based RL, whereas our method is not. Results for SAC, CURL, DrQ, and PlaNet are partially obtained from Srinivas et al. (2020); Kostrikov et al. (2020), and results for Dreamer, MuZero, and EfficientZero are obtained from Hafner et al. (2020b); Ye et al. (2021). Mean and std. deviation over 10 runs.

## Citation

```bibtex
@misc{hansen2022tdmpc,
  title={Temporal Difference Learning for Model Predictive Control},
  author={Hansen et al. (2022)},
  year={2022},
  note={arXiv:2203.04955}
}
```

- arXiv: 2203.04955