Procgen Competition Eval

---
name: procgen-competition-eval
description: Evaluates reinforcement learning agents on their ability to learn efficiently and generalize to unseen, procedurally generated environments. It measures how well algorithms adapt to novel level distributions under strict computational and timestep constraints. Use when the user wants to benchmark on Procgen Benchmark, or asks about evaluating this task. Reports mean normalized return.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2103.15332
  bibtex_key: mohanty2021procgen
  confidence: high
---

# procgen-competition-eval

> Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark — Mohanty et al. (2021) (arXiv:2103.15332, 2021)

## What this evaluates

Evaluates reinforcement learning agents on their ability to learn efficiently and generalize to unseen, procedurally generated environments. It measures how well algorithms adapt to novel level distributions under strict computational and timestep constraints.

## Datasets

- **Procgen Benchmark** — total 16; splits: train (12), test (4); repo https://github.com/openai/procgen

## Metrics

- `mean normalized return` **(primary)** — range: [0, 1]
  - The arithmetic mean of normalized returns across all evaluated environments. Normalized return is calculated per environment as (agent_return - random_agent_return) / (perfect_agent_return - random_agent_return), averaged over multiple rollouts.

## Input / output format

**Input**: Stacked visual frames (e.g., 4-frame stacks) from the procedurally generated environment, optionally augmented with additional channels like accumulated episode reward or frame differences.

**Output**: Discrete action per timestep, sampled from the agent's policy network.

## Scoring recipe

```python
def compute_mean_normalized_return(envs, agent, num_rollouts=10):
    normalized_returns = []
    for env in envs:
        returns = []
        for _ in range(num_rollouts):
            obs = env.reset()
            total_reward = 0
            done = False
            while not done:
                action = agent.act(obs)
                obs, reward, done, _ = env.step(action)
                total_reward += reward
            returns.append(total_reward)
        avg_ret = np.mean(returns)
        norm_ret = (avg_ret - random_baseline) / (perfect_baseline - random_baseline)
        normalized_returns.append(norm_ret)
    return np.mean(normalized_returns)
```

## Common pitfalls

- Training or evaluating on the hold-out (extra) environments during the training phase, which violates the generalization constraint.
- Failing to normalize returns against the official random and perfect agent baselines provided by the Procgen benchmark.
- Ignoring the strict 8M timestep limit per environment, which artificially inflates sample efficiency scores.

## Evidence (verbatim from paper)

> We used the mean normalized return to compare submissions based on a single score across multiple Procogen environments. The mean normalized rewards across multiple rollouts per environment for the top ten submission for sample-efficiency and generalization tracks are shown in 3 and 3.

## Citation

```bibtex
@misc{mohanty2021procgen,
  title={Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark},
  author={Mohanty et al. (2021)},
  year={2021},
  note={arXiv:2103.15332}
}
```

- arXiv: 2103.15332