Gazebo Simulation Eval

---
name: gazebo-simulation-eval
description: Evaluates an MPC-based autonomous driving controller's ability to perform collision avoidance and lane maneuvers (overtaking, merging, following) in dynamic environments using a physics-based simulator. It tests the controller's real-time feasibility and trajectory smoothness under varying traffic densities. Use when the user wants to benchmark on Gazebo Simulation, or asks about evaluating this task. Reports computation time.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1712.04965
  bibtex_key: babu2017mpc
  confidence: medium
---

# gazebo-simulation-eval

> Model Predictive Control for Autonomous Driving Based on Time Scaled Collision Cone — Babu et al. (2017) (arXiv:1712.04965, 2017)

## What this evaluates

Evaluates an MPC-based autonomous driving controller's ability to perform collision avoidance and lane maneuvers (overtaking, merging, following) in dynamic environments using a physics-based simulator. It tests the controller's real-time feasibility and trajectory smoothness under varying traffic densities.

## Datasets

- **Gazebo Simulation** — total ?; splits: test (-1)

## Metrics

- `computation time` **(primary)** — range: ms
  - Mean time per MPC iteration across all simulation scenarios, measured in milliseconds.
- `update rate` — range: Hz
  - Reciprocal of computation time, representing the controller's frequency in Hz.

## Input / output format

**Input**: LIDAR-derived obstacle locations and velocities, road boundary constraints, and a preferred forward velocity parameter.

**Output**: Receding horizon trajectory consisting of path (angular velocity) and forward velocity commands executed over a short planning horizon.

## Scoring recipe

```python
success = check_collision_free(traj) and check_lane_following(traj)
computation_time = mean([solve_time(mpc_iteration) for iteration in simulation])
update_rate = 1.0 / computation_time
return success, computation_time, update_rate
```

## Common pitfalls

- Evaluation is purely simulation-based without real-world validation, which may not capture sensor noise or actuator delays.
- Parameters like preferred velocity and pseudo-velocity are manually tuned per scenario rather than learned or standardized.
- Success is assessed qualitatively via trajectory plots rather than a quantitative benchmark metric.

## Evidence (verbatim from paper)

> The mean computation time for each iteration of our MPC loop observed across all the examples presented in the previous section is shown in Fig.[6]. The implementation was done in Python on a 64 bit laptop with 6GB RAM, i5 processor with 2.60 GHz clock speed. As can be seen, the proposed MPC with hierarchical path and velocity optimization can be solved at around 13ms for a scenario with 10 obstacles, resulting in an update rate of almost 77Hz.

## Citation

```bibtex
@misc{babu2017mpc,
  title={Model Predictive Control for Autonomous Driving Based on Time Scaled Collision Cone},
  author={Babu et al. (2017)},
  year={2017},
  note={arXiv:1712.04965}
}
```

- arXiv: 1712.04965