Scannet 3d Detection Eval

---
name: scannet-3d-detection-eval
description: Evaluates the ability of 3D object detectors to localize and classify indoor objects using variable-frame sparse RGB-D inputs. It probes generalization across different input modalities (reconstructed point clouds, multi-view RGB-D, monocular RGB-D) and varying numbers of input views. Use when the user wants to benchmark on ScanNet, or asks about evaluating this task. Reports mAP@0.25.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2310.05346
  bibtex_key: wu2023anyview
  confidence: high
---

# scannet-3d-detection-eval

> Anyview: Generalizable Indoor 3D Object Detection with Variable Frames — Zhenyu Wu et al. (arXiv:2310.05346, 2023)

## What this evaluates

Evaluates the ability of 3D object detectors to localize and classify indoor objects using variable-frame sparse RGB-D inputs. It probes generalization across different input modalities (reconstructed point clouds, multi-view RGB-D, monocular RGB-D) and varying numbers of input views.

## Datasets

- **ScanNet** — total 1513; splits: train (1201), val (312)

## Metrics

- `mAP@0.25` **(primary)** — range: percent
  - Mean Average Precision computed at an Intersection-over-Union (IoU) threshold of 0.25 for 3D axis-aligned bounding boxes. AP is calculated by averaging precision over recall thresholds, then averaged across all 18 object categories.
- `mAP@0.5` — range: percent
  - Same as mAP@0.25 but computed at a stricter IoU threshold of 0.5 for 3D axis-aligned bounding boxes.

## Input / output format

**Input**: Variable-frame sparse RGB-D inputs. For scene-level benchmarks: reconstructed point clouds or multi-view RGB-D images (default 50 views). For monocular: single RGB-D frame. For online: sequential RGB-D frames processed frame-by-frame.

**Output**: Set of 3D axis-aligned bounding boxes with class labels for 18 selected indoor object categories.

## Scoring recipe

```python
def compute_3d_map(preds, golds, iou_thresh=0.25):
    ap_scores = []
    for cls in classes:
        cls_preds = [p for p in preds if p['class'] == cls]
        cls_golds = [g for g in golds if g['class'] == cls]
        tp, fp = 0, 0
        for pred in sorted(cls_preds, key=lambda x: x['score'], reverse=True):
            best_iou = max(iou(pred['box'], gt['box']) for gt in cls_golds)
            if best_iou >= iou_thresh:
                tp += 1
            else:
                fp += 1
        ap = tp / (tp + fp) if (tp + fp) > 0 else 0
        ap_scores.append(ap)
    return sum(ap_scores) / len(ap_scores) * 100
```

## Common pitfalls

- Confusing ScanNet-Rec (requires pre-reconstructed point clouds) with ScanNet-MV (uses raw multi-view RGB-D), which drastically changes input availability and baseline performance.
- Assuming fixed input scales; the benchmark explicitly tests generalization across 10, 30, and 50 views, as well as streaming online detection, which breaks standard fixed-input detectors.
- Overlooking point-sampling differences in evaluation (e.g., AnyView uses 5000 points/view vs 20000/40000 for baselines), which affects computational cost but not the mAP metric itself.

## Evidence (verbatim from paper)

> ScanNet-Rec is the mainstream scene-level benchmark used by previous works, whose input data is reconstructed point cloud of a whole scene and ground-truth is generated by computing the axis-aligned bounding boxes for objects in 18 selected categories. We show the performance of different models on scene-level (Rec and MV) and monocular (SV) benchmarks in Table III. 3D object detection results (mAP@0.25 and mAP@0.5) and corresponding training/inference setting of different methods on scene-level and monocular benchmarks.

## Citation

```bibtex
@misc{wu2023anyview,
  title={Anyview: Generalizable Indoor 3D Object Detection with Variable Frames},
  author={Zhenyu Wu et al.},
  year={2023},
  note={arXiv:2310.05346}
}
```

- arXiv: 2310.05346