Neorl2 Eval

---
name: neorl2-eval
description: This benchmark evaluates offline reinforcement learning algorithms on seven near real-world environments featuring time delays, external disturbances, safety constraints, and conservative data collection. It probes whether state-of-the-art offline RL methods can improve upon sub-optimal behavior policies without online exploration, highlighting their robustness to realistic dynamics and safety limits. Use when the user wants to benchmark on NeoRL-2, or asks about evaluating this task. Reports normalized score (0-100).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2503.19267
  bibtex_key: gao2025neorl2
  confidence: high
---

# neorl2-eval

> NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios — Songyi Gao et al. (arXiv:2503.19267, 2025)

## What this evaluates

This benchmark evaluates offline reinforcement learning algorithms on seven near real-world environments featuring time delays, external disturbances, safety constraints, and conservative data collection. It probes whether state-of-the-art offline RL methods can improve upon sub-optimal behavior policies without online exploration, highlighting their robustness to realistic dynamics and safety limits.

## Datasets

- **NeoRL-2** — total ?; splits: offline_data (-1); repo https://github.com/polixir/NeoRL2

## Metrics

- `normalized score (0-100)` **(primary)** — range: [0, 100]
  - Computed by normalizing the average return of the evaluated policy against the random policy (minimum bound) and the expert SAC policy (maximum bound) for each environment. Formula: ((avg_return - random_return) / (expert_return - random_return)) * 100.

## Input / output format

**Input**: Offline trajectory datasets (states, actions, rewards) collected from sub-optimal policies (e.g., intermediate SAC checkpoints or PID controllers).

**Output**: A policy network that maps environment states to actions, evaluated via online rollouts in the target environment.

## Scoring recipe

```python
1. Collect baseline returns: random_return = run_policy(RandomPolicy(), env)
2. Collect expert return: expert_return = run_policy(ExpertSACPolicy(), env)
3. Train offline RL policy on dataset
4. Evaluate trained policy over 3 seeds: avg_return = mean([run_policy(trained_policy, env, seed=s) for s in seeds])
5. Normalize score: normalized_score = ((avg_return - random_return) / (expert_return - random_return)) * 100
6. Report mean and standard error across seeds.
```

## Common pitfalls

- Expert policy is explicitly noted as a good policy but not necessarily optimal, so the 100-point bound is an approximation.
- Model-based methods (e.g., MOPO, RAMBO) exhibit high variance across random seeds, making single-run evaluations misleading.
- Evaluation is strictly online (running the trained policy in the environment), but training must remain strictly offline using only the provided dataset.

## Evidence (verbatim from paper)

> In normalization of data scores (0-100), the scores given by expert policies are used as the maximum bound. ... We run each configuration of the hyperparameters with 3 random seeds and choose the policy at the final training stage to conduct the online test. We report the results from the best hyperparameter across 3 seeds and the 3 seeds are the same for all the algorithms.

## Citation

```bibtex
@misc{gao2025neorl2,
  title={NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios},
  author={Songyi Gao et al.},
  year={2025},
  note={arXiv:2503.19267}
}
```

- arXiv: 2503.19267