Nuscenes Unified Ad Eval

---
name: nuscenes-unified-ad-eval
description: This benchmark evaluates a unified autonomous driving model across six core tasks: 3D object detection, multi-object tracking, BEV map segmentation, 3D occupancy prediction, motion prediction, and trajectory planning. It probes the model's ability to process heterogeneous multi-modal sensor data (LiDAR, multi-view cameras, and temporal sequences) and generate accurate, decoupled predictions for perception, prediction, and planning domains. Use when the user wants to benchmark on nuScenes, or asks about evaluating this task. Reports NDS.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.01768
  bibtex_key: liu2025unilion
  confidence: high
---

# nuscenes-unified-ad-eval

> UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs — Liu et al. (2025) (arXiv:2511.01768, 2025)

## What this evaluates

This benchmark evaluates a unified autonomous driving model across six core tasks: 3D object detection, multi-object tracking, BEV map segmentation, 3D occupancy prediction, motion prediction, and trajectory planning. It probes the model's ability to process heterogeneous multi-modal sensor data (LiDAR, multi-view cameras, and temporal sequences) and generate accurate, decoupled predictions for perception, prediction, and planning domains.

## Datasets

- **nuScenes** — total 1000; splits: train (750), val (150), test (150)

## Metrics

- `NDS` **(primary)** — range: percent
  - NuScenes Detection Score, a weighted harmonic mean of detection quality metrics (mAP, attribute accuracy, etc.) normalized to a 0-100 scale.
- `mAP` — range: percent
  - mean Average Precision across all 23 nuScenes object classes, computed by averaging the area under the precision-recall curve for each class.
- `AMOTA` — range: percent
  - Association-MOTA, measures tracking accuracy by penalizing false positives, false negatives, and identity switches based on association thresholds.
- `mIoU` — range: percent
  - mean Intersection over Union for BEV map segmentation, averaging the IoU across all semantic map classes (drivable area, crosswalks, etc.).
- `RayIoU` — range: percent
  - Ray Intersection over Union for 3D occupancy prediction, computing IoU along rays from the sensor to evaluate voxel occupancy accuracy at varying distances.
- `minADE` — range: meters
  - minimum Average Displacement Error, the average L2 distance between predicted and ground truth trajectories for the best-performing prediction among multiple candidates.
- `L2` — range: meters
  - L2 distance for planning, measuring the Euclidean distance between the predicted and ground truth future ego-vehicle trajectory.
- `collision_rate` — range: percent
  - collision rate for planning, the percentage of predicted trajectories that result in a collision with static or dynamic obstacles.

## Input / output format

**Input**: Multi-modal sensor data: LiDAR point clouds voxelized into a grid (0.3m x 0.3m x 0.25m), images from 6 surrounding cameras (processed via Swin-Tiny or ResNet-50 backbones), and temporal sequences of 4 consecutive frames fed in a streaming manner.

**Output**: Task-specific predictions: 3D bounding boxes with class and attributes, track IDs and states, BEV semantic segmentation masks, 3D voxel occupancy grids, future agent trajectories (multiple candidates), and future ego-vehicle trajectory.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth):
    # Detection
    nds = compute_nuscenes_detection_score(predictions.bboxes, ground_truth.bboxes)
    map_score = compute_mean_ap(predictions.bboxes, ground_truth.bboxes, classes=23)
    # Tracking
    amota = compute_amota(predictions.tracks, ground_truth.tracks)
    # Map Segmentation
    miou = compute_mean_iou(predictions.map_masks, ground_truth.map_masks)
    # Occupancy
    rayiou = compute_ray_iou(predictions.occupancy, ground_truth.occupancy)
    # Motion Prediction
    minade_car = compute_min_ade(predictions.agent_trajectories, ground_truth.agent_trajectories, target='car')
    minade_ped = compute_min_ade(predictions.agent_trajectories, ground_truth.agent_trajectories, target='pedestrian')
    # Planning
    l2_dist = compute_l2_distance(predictions.ego_trajectory, ground_truth.ego_trajectory)
    col_rate = compute_collision_rate(predictions.ego_trajectory, ground_truth.obstacles)
    return {'NDS': nds, 'mAP': map_score, 'AMOTA': amota, 'mIoU': miou, 'RayIoU': rayiou, 'minADE': (minade_car, minade_ped), 'L2': l2_dist, 'collision_rate': col_rate}
```

## Common pitfalls

- Strictly avoid using ego-status information during training and evaluation to prevent information leakage, as some baseline methods incorrectly incorporate it.
- Motion prediction and planning tasks are only evaluated when temporal inputs (multi-frame sequences) are provided; single-frame variants do not report these metrics.
- Occupancy prediction uses the Occ3D-nuScenes validation set with distance-stratified RayIoU metrics (1m, 2m, 4m), which differ from standard voxel IoU.

## Evidence (verbatim from paper)

> Evaluation Metrics. We adopt task-specific metrics following established protocols: mean Average Precision (mAP) and NuScenes Detection Score (NDS) for 3D object detection; AMOTA for tracking performance; mean Intersection over Union (mIoU) for BEV map segmentation*[[4]]*; RayIoU for 3D occupancy prediction*[[68]]*; minimum Average Displacement Error (minADE) for motion prediction*[[66]]*; and L2 distance along with collision rate for planning evaluation*[[13]]*.

## Citation

```bibtex
@misc{liu2025unilion,
  title={UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs},
  author={Liu et al. (2025)},
  year={2025},
  note={arXiv:2511.01768}
}
```

- arXiv: 2511.01768