Libero Liberoplus Eval

---
name: libero-liberoplus-eval
description: Evaluates a Vision-Language-Action model's ability to perform precise robotic manipulation and maintain robustness under environmental perturbations. It probes spatial understanding, object manipulation, instruction following, and long-horizon task execution in both standard and perturbed simulation environments. Use when the user wants to benchmark on LIBERO, LIBERO-Plus, or asks about evaluating this task. Reports success_rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.24921
  bibtex_key: wei2026libravla
  confidence: high
---

# libero-liberoplus-eval

> Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System — Wei et al. (2026) (arXiv:2604.24921, 2026)

## What this evaluates

Evaluates a Vision-Language-Action model's ability to perform precise robotic manipulation and maintain robustness under environmental perturbations. It probes spatial understanding, object manipulation, instruction following, and long-horizon task execution in both standard and perturbed simulation environments.

## Datasets

- **LIBERO** — total ?; splits: test (-1)
- **LIBERO-Plus** — total ?; splits: test (-1)

## Metrics

- `success_rate` **(primary)** — range: percent
  - Calculated as the percentage of successful rollouts out of the total number of attempts (50 per task). The average success rate is computed across all four task suites or across the seven perturbation dimensions.
- `latency_ms` — range: other
  - Inference time measured in milliseconds per action step on an Nvidia RTX 4090 GPU.

## Input / output format

**Input**: RGB visual observations from a camera, natural language task instructions, and robot state/history context.

**Output**: Hybrid action space: a discrete coarse action token (macro-intent) followed by a continuous fine-grained action vector (micro-pose refinement).

## Scoring recipe

```python
def compute_success_rate(success_flags, num_trials=50):
    return sum(success_flags) / num_trials * 100.0

def compute_avg_success_rate(suite_results):
    return sum(suite_results) / len(suite_results)
```

## Common pitfalls

- Confusing the Zero-Shot Transfer setting (models trained on LIBERO tested on LIBERO-Plus) with the Supervised Fine-Tuning setting (models trained directly on LIBERO-Plus data).
- Misinterpreting the inverted-U performance curve for coarse bin size N as a monotonic relationship; optimal performance occurs at a specific equilibrium point (N=10), not at maximum or minimum granularity.
- Overlooking that LIBERO-Plus evaluates robustness across 7 specific perturbation dimensions (camera, lighting, background, layout, robot state, etc.) rather than standard task success.

## Evidence (verbatim from paper)

> Following standard evaluation protocols in prior works, each task is evaluated 50 times independently, 500 rollouts total for the task suites. We report the success rates on the four task suites, as well as the average success rate across all four tasks.

## Citation

```bibtex
@misc{wei2026libravla,
  title={Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System},
  author={Wei et al. (2026)},
  year={2026},
  note={arXiv:2604.24921}
}
```

- arXiv: 2604.24921