Hdp Real Vehicle Eval

---
name: hdp-real-vehicle-eval
description: Evaluates end-to-end autonomous driving planning models in real-world closed-loop scenarios, measuring success rate, trajectory stability, and safety compliance during urban driving. Use when the user wants to benchmark on Real-world driving dataset, or asks about evaluating this task. Reports closed-loop success rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.22801
  bibtex_key: zheng2026unleashing
  confidence: medium
---

# hdp-real-vehicle-eval

> Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving — Zheng et al. (2026) (arXiv:2602.22801, 2026)

## What this evaluates

Evaluates end-to-end autonomous driving planning models in real-world closed-loop scenarios, measuring success rate, trajectory stability, and safety compliance during urban driving.

## Datasets

- **Real-world driving dataset** — total ?; splits: train (20000000), val (-1), test (-1)

## Metrics

- `closed-loop success rate` **(primary)** — range: percent
  - Percentage of 200 km real-world test drives completed without safety-critical failures or navigation errors. Calculated over multiple urban scenarios.
- `stability score` — range: other
  - Composite metric measuring centering performance (lateral deviation from lane center) and speed compliance (adherence to speed limits).
- `safety score` — range: other
  - Measures collision avoidance and safe interaction with surrounding vehicles and VRUs during closed-loop driving.

## Input / output format

**Input**: Multi-modal sensor inputs and high-level navigation commands for autonomous driving.

**Output**: Planned trajectory represented as a sequence of waypoints and velocity commands.

## Scoring recipe

```python
def evaluate_closed_loop(model, scenario_dataset):
    success_count = 0
    total_scenarios = len(scenario_dataset)
    for scenario in scenario_dataset:
        state = scenario.initial_state
        for t in range(max_steps):
            action = model.predict(state)
            state = environment.step(action)
            if is_safe(state) and is_complete(state):
                success_count += 1
                break
            if is_crash(state) or is_timeout(state):
                break
    return (success_count / total_scenarios) * 100
```

## Common pitfalls

- Open-loop imitation learning metrics do not correlate with closed-loop driving performance; hybrid loss is required for closed-loop gains.
- RL post-training improves safety but can degrade overall success rate due to overly conservative behavior.
- Data scaling introduces task-specific trade-offs (e.g., improved lane-changing but degraded car-following).

## Evidence (verbatim from paper)

> We present the main results in TABLE[II](#S4.T2 "TABLE II ‣ IV-C Multimodal Capability and Data Scaling ‣ IV Imitation Learning Pre-training ‣ Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving"). HDP achieves nearly a 10x improvement in closed-loop performance compared to the base model. For the open-loop setting, during imitation pretraining, it shows that a well-designed loss function and data scaling can steadily improve performance. However, a significant improvement in the closed-loop score is observed only after applying the hybrid loss, highlighting the difference between open-loop and closed-loop metrics. The key insight is that the hybrid loss greatly enhances stability, allowing the model to have a higher probability of completing each task, thereby achieving an overall noticeable improvement.

## Citation

```bibtex
@misc{zheng2026unleashing,
  title={Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving},
  author={Zheng et al. (2026)},
  year={2026},
  note={arXiv:2602.22801}
}
```

- arXiv: 2602.22801