Bench Push Eval

---
name: bench-push-eval
description: Evaluates the transferability and performance of reinforcement learning policies for mobile robot navigation and pushing-based manipulation tasks. It probes how well policies trained in simulation handle real-world sim-to-real gaps, clutter, and sparse rewards across varying obstacle densities. Use when the user wants to benchmark on Bench-Push (Maze & Box-Delivery), or asks about evaluating this task. Reports $S_{\text{manip}}$.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.11736
  bibtex_key: zhong2025benchpush
  confidence: high
---

# bench-push-eval

> Bench-Push: Benchmarking Pushing-based Navigation and Manipulation Tasks for Mobile Robots — Zhong et al. (2025) (arXiv:2512.11736, 2025)

## What this evaluates

Evaluates the transferability and performance of reinforcement learning policies for mobile robot navigation and pushing-based manipulation tasks. It probes how well policies trained in simulation handle real-world sim-to-real gaps, clutter, and sparse rewards across varying obstacle densities.

## Datasets

- **Bench-Push (Maze & Box-Delivery)** — total ?; splits: test (-1)

## Metrics

- `$S_{\text{manip}}$` **(primary)** — range: [0, 1]
  - Task success rate: fraction of episodes where the robot successfully pushes the box to the receptacle. Values are normalized scores where higher is better.
- `$E_{\text{nav}}$` — range: [0, 1]
  - Navigation efficiency: normalized score measuring path/collision efficiency during maze traversal. Higher is better.
- `$I_{\text{nav}}$` — range: [0, 1]
  - Navigation interaction effort: normalized score measuring physical interaction intensity during navigation. Higher is better.

## Input / output format

**Input**: Overhead camera observations tracking robot pose and 3D-printed box positions at 10Hz, provided as state vectors to the deployed policy.

**Output**: Continuous control commands (velocity/steering) for the TurtleBot3 Burger.

## Scoring recipe

```python
def evaluate(episodes, task_type):
    success = 0
    eff_sum, int_sum = 0.0, 0.0
    for ep in episodes:
        if task_success(ep): success += 1
        eff_sum += normalize_efficiency(ep)
        int_sum += normalize_interaction(ep)
    n = len(episodes)
    return {
        'S_manip': success / n,
        'E_manip': eff_sum / n,
        'I_manip': int_sum / n
    }
```

## Common pitfalls

- Physical testbed scores are consistently lower than simulation due to sim-to-real gaps (localization noise, control imperfections, box slipping), which should not be mistaken for policy failure.
- Metrics degrade monotonically with increased obstacle/box counts; comparing across different clutter levels without normalization is misleading.
- Interaction effort and efficiency are normalized scores, not raw physical units, so cross-environment absolute comparisons require caution.

## Evidence (verbatim from paper)

> TABLE IV: Results for Box-Delivery in the physical testbed and Bench-Push simulations.  $\uparrow$  indicates higher is better.  ... Smanip↑ Emanip↑ Imanip↑

## Citation

```bibtex
@misc{zhong2025benchpush,
  title={Bench-Push: Benchmarking Pushing-based Navigation and Manipulation Tasks for Mobile Robots},
  author={Zhong et al. (2025)},
  year={2025},
  note={arXiv:2512.11736}
}
```

- arXiv: 2512.11736