Bench2drive Speed Eval

---
name: bench2drive-speed-eval
description: Evaluates autonomous driving policies' ability to follow explicit user commands for target speed and overtake/follow behaviors in closed-loop simulations. It measures how well models track desired speeds and execute passing maneuvers while maintaining safety, comfort, and traffic compliance. Use when the user wants to benchmark on Bench2Drive-Speed, or asks about evaluating this task. Reports Speed-Adherence Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.25672
  bibtex_key: [REDACTED]
  confidence: high
---

# bench2drive-speed-eval

> Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving — Shao et al. (2026) (arXiv:2603.25672, 2026)

## What this evaluates

Evaluates autonomous driving policies' ability to follow explicit user commands for target speed and overtake/follow behaviors in closed-loop simulations. It measures how well models track desired speeds and execute passing maneuvers while maintaining safety, comfort, and traffic compliance.

## Datasets

- **Bench2Drive-Speed** — total 48; splits: test (48); repo https://github.com/Thinklab-SJTU/Bench2Drive-Speed

## Metrics

- `Speed-Adherence Score` **(primary)** — range: percent
  - Quantifies policy fidelity to target-speed commands while jointly assessing safety, comfort, and traffic compliance over the evaluation route.
- `Overtake Score` — range: percent
  - Measures the success rate of executing overtake commands relative to follow commands, penalizing safety violations and route failures.

## Input / output format

**Input**: Ego vehicle state, RGB camera inputs, target waypoint (goal), and driving commands (target speed and overtake/follow instruction) concatenated as model input.

**Output**: Trajectory and control actions; only the trajectory branch output is utilized for closed-loop execution.

## Scoring recipe

```python
def compute_speed_adherence_score(trajectory, target_speed, route):
    speed_error = mean_absolute_error(trajectory.speed, target_speed)
    penalty = safety_violations(trajectory) + comfort_penalty(trajectory) + traffic_penalty(trajectory)
    return max(0, 100 - (speed_error * weight + penalty))

def compute_overtake_score(trajectory, command, route):
    if command == 'overtake':
        success = check_overtake_maneuver(trajectory) and not safety_violations(trajectory)
        return 100 if success else 0
    return 0
```

## Common pitfalls

- Overtaking commands often trigger aggressive maneuvers that increase collision risks, leading to safety violations that reduce route completion and artificially lower the overtake score.
- Virtual target-speed annotation using long extrapolation horizons introduces monotonic trend uncertainty and amplified noise, reducing speed adherence stability compared to short horizons.
- Models trained without explicit speed commands default to single-policy behaviors and cannot follow user-specified target speeds or overtake/follow instructions.

## Evidence (verbatim from paper)

> Table 5: Speed-Adherence Score and Overtake Score on 48 evaluation routes of Bench2Drive-Speed. Metrics are reported for All(A), Easy (E), Medium (M), and Hard (H).

## Citation

```bibtex
@misc{shao2026bench2drivespeed,
  title={Can Users Specify Driving Speed? Bench2Drive-Speed: Benchmark and Baselines for Desired-Speed Conditioned Autonomous Driving},
  author={Shao et al. (2026)},
  year={2026},
  note={arXiv:2603.25672}
}
```

- arXiv: 2603.25672