Habitat Gs Eval

---
name: habitat-gs-eval
description: Evaluates embodied agents' navigation capabilities in photorealistic 3D Gaussian Splatting environments versus traditional mesh-based simulators. It probes cross-domain generalization, visual robustness, and human-aware collision avoidance in dynamic scenes. Use when the user wants to benchmark on InteriorGS + Real-world GS, Habitat-Matterport 3D (HM3D), AnimatableGaussians, or asks about evaluating this task. Reports SR, SPL.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.12626
  bibtex_key: xia2026habitatgs
  confidence: high
---

# habitat-gs-eval

> Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting — Xia et al. (2026) (arXiv:2604.12626, 2026)

## What this evaluates

Evaluates embodied agents' navigation capabilities in photorealistic 3D Gaussian Splatting environments versus traditional mesh-based simulators. It probes cross-domain generalization, visual robustness, and human-aware collision avoidance in dynamic scenes.

## Datasets

- **InteriorGS + Real-world GS** — total 120; splits: train (100), test (20)
- **Habitat-Matterport 3D (HM3D)** — total 120; splits: train (100), test (20)
- **AnimatableGaussians** — total 6; splits: train (3), test (3)

## Metrics

- `SR` **(primary)** — range: percent
  - Fraction of episodes where the agent reaches the goal within a predefined distance threshold.
- `SPL` **(primary)** — range: percent
  - SR × (ℓ*/ℓ), where ℓ* is the geodesic shortest-path length and ℓ is the agent's actual path length.
- `DTG` — range: other
  - Euclidean distance between the agent and the goal at episode termination.
- `CR` — range: percent
  - Fraction of steps in an episode where the agent collides with an obstacle or avatar.
- `PSI` — range: other
  - Average degree to which the agent enters the 1.0 m personal-space radius around each avatar.

## Input / output format

**Input**: 256×256 RGB and depth observations per timestep.

**Output**: Sequential navigation actions (e.g., forward, turn left/right) to reach a point-goal.

## Scoring recipe

```python
def compute_metrics(trajectory, goal_pos, shortest_path_len, collision_steps, personal_space_intrusions):
    sr = 1.0 if distance(trajectory[-1], goal_pos) <= threshold else 0.0
    spl = sr * (shortest_path_len / len(trajectory)) if len(trajectory) > 0 else 0.0
    dtg = distance(trajectory[-1], goal_pos)
    cr = collision_steps / len(trajectory)
    psi = sum(personal_space_intrusions) / len(personal_space_intrusions)
    return {'SR': sr, 'SPL': spl, 'DTG': dtg, 'CR': cr, 'PSI': psi}
```

## Common pitfalls

- Test sets for mesh and GS scenes are drawn from disjoint physical spaces, so results measure cross-domain generalization rather than representation fidelity on identical geometry.
- VLM quality assessment uses blind, randomized batches of 5 GS and 5 mesh images to prevent source bias.
- Fixed training budget (5×10^7 steps) means GS-only agents may not fully converge, affecting direct performance comparisons with mesh-only baselines.

## Evidence (verbatim from paper)

> We adopt standard embodied navigation metrics. Success Rate (SR) measures the fraction of episodes in which the agent reaches the goal within a distance threshold. Success weighted by Path Length (SPL) jointly captures success and path efficiency: SPL = SR × (ℓ*/ℓ), where ℓ* is the geodesic shortest-path length and ℓ the agent’s actual path length. Distance to Goal (DTG) records the Euclidean distance between the agent and the goal at episode termination.

## Citation

```bibtex
@misc{xia2026habitatgs,
  title={Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting},
  author={Xia et al. (2026)},
  year={2026},
  note={arXiv:2604.12626}
}
```

- arXiv: 2604.12626