Being H05 Robot Eval

---
name: being-h05-robot-eval
description: Evaluates cross-embodiment generalization and manipulation capabilities of Vision-Language-Action models across heterogeneous real robots and simulation benchmarks. Probes spatial reasoning, long-horizon planning, bimanual coordination, and zero-shot transfer to unseen task-embodiment pairs. Use when the user wants to benchmark on Real-robot task suite, LIBERO, RoboCasa, or asks about evaluating this task. Reports success rate (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.12993
  bibtex_key: luo2026beingh05
  confidence: high
---

# being-h05-robot-eval

> Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization — Hao Luo et al. (2026) (arXiv:2601.12993, 2026)

## What this evaluates

Evaluates cross-embodiment generalization and manipulation capabilities of Vision-Language-Action models across heterogeneous real robots and simulation benchmarks. Probes spatial reasoning, long-horizon planning, bimanual coordination, and zero-shot transfer to unseen task-embodiment pairs.

## Datasets

- **Real-robot task suite** — total ?; splits: test (-1)
- **LIBERO** — total ?; splits: test (-1)
- **RoboCasa** — total ?; splits: test (-1)

## Metrics

- `success rate (%)` **(primary)** — range: percent
  - Binary success/failure per trial based on pre-defined objective criteria. Reported as the mean success rate over K trials (K=20 for real robots, 500 per suite for LIBERO, 50 per task for RoboCasa).

## Input / output format

**Input**: Multi-view RGB images (224x224 resolution), natural language task instructions, and robot proprioceptive/sensory observations.

**Output**: Unified action space tokens representing robot motor commands (e.g., joint velocities/positions, gripper states).

## Scoring recipe

```python
def compute_success_rate(model, tasks, trials_per_task=20):
    total_successes = 0
    total_trials = 0
    for task in tasks:
        for config in task.predefined_layouts:
            for trial in range(trials_per_task):
                obs = get_observation(config, trial)
                action = model.predict(obs)
                success = check_objective_criteria(action, config)
                total_successes += success
                total_trials += 1
    return (total_successes / total_trials) * 100
```

## Common pitfalls

- Operator bias can skew real-robot results; the paper enforces a blind black-box inference server to prevent this.
- Success criteria must be strictly pre-defined and objective, as human operators record binary outcomes without knowing the model identity.
- LIBERO and RoboCasa use different trial counts (500 vs 50) and scene splits, so averaging must follow each benchmark's specific protocol.

## Evidence (verbatim from paper)

> We report the mean success rate over 50 evaluation episodes per task. Being-H0.5(specialist) reaches a 98.9% average success rate on LIBERO, with strong performance across all suites including LIBERO-Long (97.4%).

## Citation

```bibtex
@misc{luo2026beingh05,
  title={Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization},
  author={Hao Luo et al. (2026)},
  year={2026},
  note={arXiv:2601.12993}
}
```

- arXiv: 2601.12993