Nuscenes 3d Detection Eval

---
name: nuscenes-3d-detection-eval
description: Evaluates the capability of multi-view 3D object detection models to accurately localize and classify objects in autonomous driving scenes using camera inputs. It measures detection accuracy alongside computational efficiency and inference latency to assess real-time deployment feasibility. Use when the user wants to benchmark on NuScenes, or asks about evaluating this task. Reports NDS.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.13586
  bibtex_key: [REDACTED]
  confidence: high
---

# nuscenes-3d-detection-eval

> Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning — Danish Nazir et al. (2026) (arXiv:2604.13586, 2026)

## What this evaluates

Evaluates the capability of multi-view 3D object detection models to accurately localize and classify objects in autonomous driving scenes using camera inputs. It measures detection accuracy alongside computational efficiency and inference latency to assess real-time deployment feasibility.

## Datasets

- **NuScenes** — total ?; splits: train (700), val (150)

## Metrics

- `mAP` — range: [0, 1]
  - Mean Average Precision computed over S object classes and U 3D bounding box thresholds: mAP = 1/(S*U) * sum_{s in S} sum_{u in U} AP_{u,s}.
- `NDS` **(primary)** — range: [0, 1]
  - NuScenes Detection Score combines mAP with 5 true positive error metrics (ATE, ASE, AOE, AVE, AAE): NDS = 0.5 * mAP + 0.1 * sum_{w=1 to 5} (1 - min(1, mTP_w)), where mTP_w is the mean error over all classes.
- `GFLOPs` — range: other
  - Total number of floating point arithmetic operations required to perform a single forward pass through the multi-view 3D object detection pipeline.
- `Average Rank` — range: other
  - Row-wise average of ranks across all evaluation and efficiency metrics for each method. Higher is better for mAP/NDS, lower is better for error metrics, GFLOPs, and latency.
- `Latency` — range: other
  - Wall-clock time for a single forward pass, computed as the sum of encoder, FPN, and decoder processing times: tau = tau^E + tau^FPN + tau^D.

## Input / output format

**Input**: Six calibrated camera images per scene (320x800 or 800x1600 resolution) captured at 2Hz, representing a full 360° field of view around the ego vehicle.

**Output**: 3D bounding boxes for 10 common object classes, including attributes, velocity, and orientation predictions.

## Scoring recipe

```python
def compute_nuscenes_metrics(predictions, ground_truth):
    mAP = compute_mean_average_precision(predictions, ground_truth)
    tp_errors = compute_tp_errors(predictions, ground_truth)  # ATE, ASE, AOE, AVE, AAE
    mTP_w = [mean(tp_errors[w][s] for s in classes) for w in range(5)]
    NDS = 0.5 * mAP + 0.1 * sum(1 - min(1, m) for m in mTP_w)
    latency = time_encoder + time_fpn + time_decoder
    return mAP, NDS, latency, tp_errors
```

## Common pitfalls

- The official NuScenes test server is non-functional, so results must be reported on the validation split rather than the test split.
- Latency and GFLOPs comparisons require matching input resolutions and backbone configurations to ensure fair evaluation.
- Training is conducted on H100 GPUs, but latency evaluation must be performed on GV100 GPUs to simulate real-world deployment scenarios.

## Evidence (verbatim from paper)

> The NuScenes detection score (NDS) is the primary evaluation metric of the NuScenes benchmark. It combines mAP with W=5 true positive (TP) error metrics, including average translation error (ATE), average scale error (ASE), average orientation error (AOE), average velocity error (AVE), and average attribute error (AAE) to jointly evaluate the localization, scale, orientation, velocity, and attribute accuracy of the predicted 3D object bounding boxes. Using ([4]) and ([5]), the NDS is defined as NDS = 1/2 mAP + 1/10 sum_{w in W} (1 - min(1, mTP_w)).

## Citation

```bibtex
@misc{nazir2026efficientmultiview,
  title={Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning},
  author={Danish Nazir et al. (2026)},
  year={2026},
  note={arXiv:2604.13586}
}
```

- arXiv: 2604.13586