Wilddet3d Eval

---
name: wilddet3d-eval
description: Evaluates open-vocabulary monocular 3D object detection across diverse real-world and synthetic scenes. It probes the model's ability to localize and regress 3D bounding boxes using text or geometric prompts, measuring generalization to unseen categories and datasets with and without depth cues. Use when the user wants to benchmark on WildDet3D-Bench, Omni3D, Argoverse 2, ScanNet, Stereo4D, or asks about evaluating this task. Reports AP_3D.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.08626
  bibtex_key: huang2026wilddet3d
  confidence: high
---

# wilddet3d-eval

> WildDet3D: Scaling Promptable 3D Detection in the Wild — Huang et al. (2026) (arXiv:2604.08626, 2026)

## What this evaluates

Evaluates open-vocabulary monocular 3D object detection across diverse real-world and synthetic scenes. It probes the model's ability to localize and regress 3D bounding boxes using text or geometric prompts, measuring generalization to unseen categories and datasets with and without depth cues.

## Datasets

- **WildDet3D-Bench** — total ?; splits: test (-1)
- **Omni3D** — total ?; splits: test (-1)
- **Argoverse 2** — total ?; splits: test (-1)
- **ScanNet** — total ?; splits: test (-1)
- **Stereo4D** — total 383; splits: test (383)

## Metrics

- `AP_3D` **(primary)** — range: percent
  - Average Precision computed over 10 IoU thresholds [0.05:0.50:0.05] for Omni3D, or center-distance matching thresholds [0.50:1.00:0.05] for WildDet3D-Bench and Stereo4D. A prediction matches a ground truth if the 3D center distance is within a fraction of the object radius.
- `ODS` — range: percent
  - Open Detection Score combining detection accuracy and localization errors: ODS = (3·AP + (1 - mATE) + (1 - mAOE) + (1 - mASE)) / 6, where mATE, mAOE, and mASE denote mean translation, orientation, and scale errors.

## Input / output format

**Input**: Monocular RGB image (resized to 1008×1008), optionally paired with a depth map. Prompt is either a text category name or an oracle 2D bounding box.

**Output**: 3D bounding box predictions comprising center coordinates, dimensions, orientation, and a confidence score.

## Scoring recipe

```python
def compute_ap_3d(preds, gts, match_type='center_distance'):
    tp, fp = [], []
    for gt in gts:
        matched = False
        for pred in preds:
            if match_type == 'center_distance':
                dist = l2_distance(pred.center, gt.center)
                if dist <= pred.radius * threshold:
                    matched = True
                    break
            elif match_type == 'iou':
                if compute_3d_iou(pred, gt) >= threshold:
                    matched = True
                    break
        tp.append(matched)
        fp.append(not matched)
    return calculate_ap(tp, fp)
```

## Common pitfalls

- For Omni3D, predictions matching 'ignore' annotated objects must be treated as neutral, not false positives.
- WildDet3D-Bench and Stereo4D use center-distance matching relative to object radius, not standard 3D IoU.
- On WildDet3D-Bench, overlaps with 2D-annotated objects lacking valid 3D boxes are neutral per the LVIS federated protocol.

## Evidence (verbatim from paper)

> For Omni3D, we follow the standard protocol and report Average Precision $(\mathrm{AP}_{3\mathrm{D}})$ at 3D IoU thresholds $[0.05:0.50:0.05]$ (10 thresholds). Objects annotated as ignore (e.g., invalid 3D annotation, heavy truncation) are excluded from both the positive ground-truth set and the false-positive count: a prediction matching an ignored object is treated as neutral (see Section 2.4). For zero-shot transfer on Argoverse 2 and ScanNet, we follow the 3D-MOOD protocol [58] and report the Open Detection Score (ODS), which combines AP with three error metrics into a unified score: ODS = $(3\cdot \mathrm{AP} + (1 - \mathrm{mATE}) + (1 - \mathrm{mAOE}) + (1 - \mathrm{mASE})) / 6$

## Citation

```bibtex
@misc{huang2026wilddet3d,
  title={WildDet3D: Scaling Promptable 3D Detection in the Wild},
  author={Huang et al. (2026)},
  year={2026},
  note={arXiv:2604.08626}
}
```

- arXiv: 2604.08626