Helms Exploration Eval

---
name: helms-exploration-eval
description: Evaluates the exploration efficiency and adaptability of a robot planner in unknown environments. It measures how well the system covers free space while minimizing travel distance and adapting to natural language preferences. Use when the user wants to benchmark on Simulated dungeon environments, Indoor office environment (130m x 100m), or asks about evaluating this task. Reports Travel Distance.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2503.07006
  bibtex_key: liao2025helms
  confidence: high
---

# helms-exploration-eval

> HELM: Human-Preferred Exploration with Language Models — Liao et al. (2025) (arXiv:2503.07006, 2025)

## What this evaluates

Evaluates the exploration efficiency and adaptability of a robot planner in unknown environments. It measures how well the system covers free space while minimizing travel distance and adapting to natural language preferences.

## Datasets

- **Simulated dungeon environments** — total 100; splits: test (100)
- **Indoor office environment (130m x 100m)** — total 1; splits: test (1)

## Metrics

- `Travel Distance` **(primary)** — range: other
  - Total Euclidean length of the robot's trajectory from start until >99% of the free area is covered.
- `Replanning Time` — range: other
  - Wall-clock time required for the planner to generate a new trajectory upon receiving a preference update or sensor input.

## Input / output format

**Input**: Natural language prompt specifying task or human preferences, combined with real-time onboard sensor data (LiDAR) and partial map observations.

**Output**: Sequential robot motion commands forming an exploration trajectory.

## Scoring recipe

```python
def compute_travel_distance(trajectory, ground_truth_free_area):
    covered = 0.0
    total_dist = 0.0
    for i in range(1, len(trajectory)):
        total_dist += euclidean_dist(trajectory[i-1], trajectory[i])
        covered += newly_explored_area(trajectory[i])
        if covered / ground_truth_free_area > 0.99:
            return total_dist
    return total_dist
```

## Common pitfalls

- Exploration completion is defined as >99% coverage, not 100%, to allow for sensor/model tolerances.
- The 'Optimal' baseline approximates the true optimum via TSP on sampled nodes covering ground-truth frontiers, not an exact solver.
- Results report mean and variance across 100 environments, so single-environment scores are not directly comparable to the paper's headline numbers.

## Evidence (verbatim from paper)

> We compare the travel distance of HELM to complete the exploration with several conventional planners... The mean and variance of the trajectory lengths required for full exploration are summarized in Table I. In our experiments, exploration is considered complete once more than 99% of the free area has been covered, allowing for minor tolerances in error.

## Citation

```bibtex
@misc{liao2025helms,
  title={HELM: Human-Preferred Exploration with Language Models},
  author={Liao et al. (2025)},
  year={2025},
  note={arXiv:2503.07006}
}
```

- arXiv: 2503.07006