L2rpn 2020 Eval

---
name: l2rpn-2020-eval
description: Evaluates an agent's ability to safely manage power grid topology under unexpected line failures and fluctuating renewable energy generation. It probes robustness to sudden grid attacks and adaptability to changing energy mix proportions over a full year of seasonal scenarios. Use when the user wants to benchmark on Grid2Op (NeurIPS 2020 L2RPN), or asks about evaluating this task. Reports total_reward.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2106.15200
  bibtex_key: zhou2021actionset
  confidence: high
---

# l2rpn-2020-eval

> Action Set Based Policy Optimization for Safe Power Grid Management — Bo Zhou et al. (2021) (arXiv:2106.15200, 2021)

## What this evaluates

Evaluates an agent's ability to safely manage power grid topology under unexpected line failures and fluctuating renewable energy generation. It probes robustness to sudden grid attacks and adaptability to changing energy mix proportions over a full year of seasonal scenarios.

## Datasets

- **Grid2Op (NeurIPS 2020 L2RPN)** — total 24; splits: test (24)

## Metrics

- `total_reward` **(primary)** — range: other
  - Sum of re-scaled rewards across 24 unseen monthly scenarios. Higher values indicate better grid management performance.

## Input / output format

**Input**: Grid state vector (1266 dimensions for robustness, ~3800 for adaptability) encoding substations, power lines, generators, and loads.

**Output**: Probability distribution over topological actions (~66,918 for robustness, ~200,000+ for adaptability), from which K=100 action candidates are sampled and selected via search-based planning.

## Scoring recipe

```python
total_score = 0.0
for scenario in unseen_monthly_scenarios: # 24 total
    env = load_scenario(scenario)
    episode_reward = 0.0
    while not env.done:
        action = policy.select_action(env.state)
        next_state, r, done, info = env.step(action)
        episode_reward += r
    total_score += re_scale_reward(episode_reward)
return total_score
```

## Common pitfalls

- The action space is extremely large (~66k–200k topological configurations), so standard RL without action-set constraints will fail to explore effectively.
- Rewards are re-scaled per environment before summation, meaning raw episode returns cannot be directly compared across tracks or without normalization.
- Evaluation covers 24 unseen monthly scenarios, so performance is highly sensitive to seasonal load and renewable generation patterns not seen during training.

## Evidence (verbatim from paper)

> Each submitted agent is tested in 24 unseen scenarios that cover every month of the year. The reward in each environment is re-scaled, and the total reward of 24 environments is used for ranking.

## Citation

```bibtex
@misc{zhou2021actionset,
  title={Action Set Based Policy Optimization for Safe Power Grid Management},
  author={Bo Zhou et al. (2021)},
  year={2021},
  note={arXiv:2106.15200}
}
```

- arXiv: 2106.15200