Powrl Eval

---
name: powrl-eval
description: Evaluates reinforcement learning agents for real-time power grid topology control under adversarial attacks and dynamic loads. It probes the agent's ability to maintain grid stability, minimize operational costs, and avoid blackouts across multiple challenging scenarios. Use when the user wants to benchmark on L2RPN NeurIPS 2020 (Robustness track) Offline, L2RPN NeurIPS 2020 (Robustness track) Online, L2RPN WCCI 2020 Offline, or asks about evaluating this task. Reports survival steps, scenario score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2212.02397
  bibtex_key: chauhan2022powrl
  confidence: high
---

# powrl-eval

> PowRL: A Reinforcement Learning Framework for Robust Management of Power Networks — Chauhan et al. (2022) (arXiv:2212.02397, 2022)

## What this evaluates

Evaluates reinforcement learning agents for real-time power grid topology control under adversarial attacks and dynamic loads. It probes the agent's ability to maintain grid stability, minimize operational costs, and avoid blackouts across multiple challenging scenarios.

## Datasets

- **L2RPN NeurIPS 2020 (Robustness track) Offline** — total ?; splits: test (24)
- **L2RPN NeurIPS 2020 (Robustness track) Online** — total ?; splits: test (24)
- **L2RPN WCCI 2020 Offline** — total ?; splits: test (10)

## Metrics

- `survival steps` **(primary)** — range: other
  - Total number of 5-minute time steps an agent successfully operates before a blackout or failure occurs.
- `scenario score` **(primary)** — range: other
  - Composite metric combining operation cost and losses due to blackouts; lower values indicate better performance.
- `fully survive count` — range: other
  - Number of scenarios out of the total where the agent survives all time steps without failure.

## Input / output format

**Input**: Power network state at each 5-minute time step, including power flow, generation/consumption amounts, and network architecture (reduced IEEE-118 system with 22 generators, 36 substations, 37 loads, 59 lines).

**Output**: Discrete topology control action from a reduced action space of 208 possible actions (2 illegal), selected only when line overflow exceeds a safety threshold.

## Scoring recipe

```python
def evaluate_agent(agent, scenario):
    total_steps = len(scenario)
    survival_steps = 0
    operation_cost = 0
    blackout_loss = 0
    for t in range(total_steps):
        state = scenario.get_state(t)
        if agent.should_act(state):
            action = agent.select_action(state)
            next_state, reward, done = env.step(action)
            operation_cost += reward.cost
            if done:
                blackout_loss += penalty
                break
        survival_steps += 1
    scenario_score = operation_cost + blackout_loss
    return survival_steps, scenario_score
```

## Common pitfalls

- Online test dataset is hidden from participants, requiring Codalab submission for direct evaluation.
- Baseline source code for some competitors (e.g., lujixiang) is broken or unavailable, hindering direct reproduction.
- Action space reduction (208 vs 70k) means performance is not directly comparable to agents using the full action space without normalization.

## Evidence (verbatim from paper)

> Each scenario is evaluated/scored based on the operation cost and the losses due to blackout, hence we have used both survival steps and scenario score as the evaluation criterion.

## Citation

```bibtex
@misc{chauhan2022powrl,
  title={PowRL: A Reinforcement Learning Framework for Robust Management of Power Networks},
  author={Chauhan et al. (2022)},
  year={2022},
  note={arXiv:2212.02397}
}
```

- arXiv: 2212.02397