Metaworld Eval

---
name: metaworld-eval
description: Evaluates reinforcement learning agents on their ability to learn and generalize across multiple robotic manipulation tasks. It tests multi-task RL by measuring performance on a shared set of training tasks, and meta-RL by measuring rapid adaptation to completely unseen tasks from the same distribution. Use when the user wants to benchmark on Meta-World, or asks about evaluating this task. Reports average expected return.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1910.10897
  bibtex_key: yu2019metaworld
  confidence: high
---

# metaworld-eval

> Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning — Yu et al. (2019) (arXiv:1910.10897, 2019)

## What this evaluates

Evaluates reinforcement learning agents on their ability to learn and generalize across multiple robotic manipulation tasks. It tests multi-task RL by measuring performance on a shared set of training tasks, and meta-RL by measuring rapid adaptation to completely unseen tasks from the same distribution.

## Datasets

- **Meta-World** — total 50; splits: meta-train (-1), meta-test (-1); repo https://github.com/rlworkgroup/metaworld

## Metrics

- `average expected return` **(primary)** — range: other
  - The sum of discounted rewards over the episode horizon: E_pi[sum_{t=0}^{T} gamma^t R_t(s_t, a_t)]. Evaluated as the average across tasks for multi-task RL, or on held-out tasks for meta-RL.

## Input / output format

**Input**: State vector s (fixed dimensionality across tasks), optional task encoding z (for multi-task RL), and environment dynamics.

**Output**: Action a sampled from policy pi(a|s) or pi(a|s,z).

## Scoring recipe

```python
def compute_return(policy, env, horizon, gamma):
    total_reward = 0
    s = env.reset()
    for t in range(horizon):
        a = policy(s)
        s, r, done, _ = env.step(a)
        total_reward += (gamma ** t) * r
        if done: break
    return total_reward

# Multi-task eval
avg_return = mean([compute_return(policy, env_t, H, gamma) for env_t in train_envs])

# Meta-RL eval
adapted_policy = adapt(policy, new_env, few_shot_data)
meta_test_return = compute_return(adapted_policy, new_env, H, gamma)
```

## Common pitfalls

- Multi-task RL is evaluated solely on training tasks with no held-out test set, which can overestimate generalization.
- State dimensionality is fixed across all tasks even when some tasks use fewer objects, requiring policies to handle unused state coordinates.
- Meta-RL evaluation requires adaptation with a very small number of samples; using too many adaptation steps violates the rapid-adaptation premise.

## Evidence (verbatim from paper)

> The goal of multi-task RL is to learn a single, task-conditioned policy π(a|s,z), where z indicates an encoding of the task ID. This policy should maximize the average expected return across all tasks from the task distribution p(T), given by E_{T~p(T)}[E_π[∑_{t=0}^{T}γ^t R_t(s_t,a_t)]]. ... At meta-test time, a new task T_j~p(T) is sampled that was not seen during meta-training, and the meta-trained policy must quickly adapt to this task to achieve the highest return with a small number of samples.

## Citation

```bibtex
@misc{yu2019metaworld,
  title={Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning},
  author={Yu et al. (2019)},
  year={2019},
  note={arXiv:1910.10897}
}
```

- arXiv: 1910.10897