Minivla Nav V1 Eval

---
name: minivla-nav-v1-eval
description: Language-conditioned robot navigation in continuous differential-drive settings. It probes a model's ability to process multi-modal observations (RGB, depth) and language instructions to output continuous control actions to reach a target object within a specified distance. Use when the user wants to benchmark on MiniVLA-Nav v1, or asks about evaluating this task. Reports success.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2605.00397
  bibtex_key: al-bustami2026minivlanav
  confidence: high
---

# minivla-nav-v1-eval

> MiniVLA-Nav v1: A Multi-Scene Simulation Dataset for Language-Conditioned Robot Navigation — Al-Bustami et al. (2026) (arXiv:2605.00397, 2026)

## What this evaluates

Language-conditioned robot navigation in continuous differential-drive settings. It probes a model's ability to process multi-modal observations (RGB, depth) and language instructions to output continuous control actions to reach a target object within a specified distance.

## Datasets

- **MiniVLA-Nav v1** — total 1174; splits: test (-1)

## Metrics

- `success` **(primary)** — range: [0, 1]
  - Fraction of episodes where the robot's final position is within 1.0 m of the target object's centroid and remains stationary for ≥5 consecutive steps.

## Input / output format

**Input**: Natural-language instruction and a time-series stream of front-facing RGB and depth observations at 60 Hz.

**Output**: Sequence of continuous control actions (linear velocity v ∈ [0,1] m/s, angular velocity ω ∈ [-1.5,1.5] rad/s), or quantized 7×7 discrete tokens for VLA models.

## Scoring recipe

```python
def compute_success(predictions, gold):
    successes = 0
    for pred, gold_ep in zip(predictions, gold):
        final_pos = pred[-1].position
        target_pos = gold_ep.target_position
        if distance(final_pos, target_pos) <= 1.0:
            if is_stationary_for_n_steps(pred, n=5):
                successes += 1
    return successes / len(predictions)
```

## Common pitfalls

- The success criterion requires the robot to be stationary for ≥5 consecutive steps after reaching the target, not just reaching the 1.0 m radius.
- Episodes are terminated early on collision (stall detection) or timeout (1000 steps), so models must handle failure conditions gracefully rather than just optimizing for final position.
- Action space is quantized to 49 discrete tokens for VLA models, but the ground truth and evaluation assume continuous control; mismatched discretization can cause evaluation errors.

## Evidence (verbatim from paper)

> Given a natural-language instruction $\ell$ and a stream of front-facing observations $o_{t}\=(\mathbf{I}_{t}^{\text{RGB}},\mathbf{D}_{t})$, the robot must output a sequence of actions $a_{t}\=(v_{t},\omega_{t})$ such that $\|p_{T}-p_{g}\|\leq r_{\text{success}}\=1.0\,\text{m}$... Success: robot within $r_{\text{success}}$ and stationary for $\geq 5$ consecutive steps (stopped-hold criterion).

## Citation

```bibtex
@misc{al-bustami2026minivlanav,
  title={MiniVLA-Nav v1: A Multi-Scene Simulation Dataset for Language-Conditioned Robot Navigation},
  author={Al-Bustami et al. (2026)},
  year={2026},
  note={arXiv:2605.00397}
}
```

- arXiv: 2605.00397