Mujoco Continuous Control Eval

---
name: mujoco-continuous-control-eval
description: Evaluates deep reinforcement learning algorithms on continuous control tasks. It probes capabilities in handling high-dimensional state/action spaces, partial observability, sensor noise, delayed actions, and hierarchical decision-making across physics-based simulations. Use when the user wants to benchmark on DeepMind Control Suite (MuJoCo Tasks), or asks about evaluating this task. Reports reward.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1604.06778
  bibtex_key: duan2016benchmarking
  confidence: high
---

# mujoco-continuous-control-eval

> Benchmarking Deep Reinforcement Learning for Continuous Control — Duan et al. (2016) (arXiv:1604.06778, 2016)

## What this evaluates

Evaluates deep reinforcement learning algorithms on continuous control tasks. It probes capabilities in handling high-dimensional state/action spaces, partial observability, sensor noise, delayed actions, and hierarchical decision-making across physics-based simulations.

## Datasets

- **DeepMind Control Suite (MuJoCo Tasks)** — total 31; splits: test (-1); repo https://github.com/rllab/rllab

## Metrics

- `reward` **(primary)** — range: other
  - Mean cumulative reward over a fixed number of evaluation episodes, averaged across multiple random seeds.

## Input / output format

**Input**: Continuous state vectors (e.g., positions, velocities, joint angles) and optional range sensor readings.

**Output**: Continuous action vectors applied to the physics simulator at each timestep.

## Scoring recipe

```python
def evaluate(policy, env, num_episodes, num_seeds):
    returns = []
    for seed in range(num_seeds):
        env.seed(seed)
        for _ in range(num_episodes):
            obs = env.reset()
            ep_return = 0
            done = False
            while not done:
                action = policy(obs)
                obs, reward, done, _ = env.step(action)
                ep_return += reward
            returns.append(ep_return)
    return np.mean(returns)
```

## Common pitfalls

- RL benchmarks typically do not use fixed train/validation/test splits; performance is reported as average return over random seeds and episodes.
- Tasks include specific penalty terms (e.g., for excessive control or falling) that heavily influence early learning and final performance, requiring careful comparison baselines.

## Evidence (verbatim from paper)

> The tasks in the presented benchmark can be divided into four categories: basic tasks, locomotion tasks, partially observable tasks, and hierarchical tasks. We briefly describe them in this section. More detailed specifications are given in the supplementary materials and in the source code. The goal for all the tasks is to move forward as quickly as possible. These tasks are more challenging than the basic tasks due to high degrees of freedom. In addition, a great amount of exploration is needed to learn to move forward without getting stuck at local optima. Since we penalize for excessive controls as well as falling over, during the initial stage of learning, when the robot is not yet able to move forward for a sufficient distance without falling, apparent local optima exist including staying at the origin or diving forward slowly.

## Citation

```bibtex
@misc{duan2016benchmarking,
  title={Benchmarking Deep Reinforcement Learning for Continuous Control},
  author={Duan et al. (2016)},
  year={2016},
  note={arXiv:1604.06778}
}
```

- arXiv: 1604.06778