Medical Object Detection Eval

---
name: medical-object-detection-eval
description: Evaluates the ability of object detection models to localize and classify medical structures in 3D imaging data without manual hyperparameter tuning. It probes generalization across diverse anatomical regions and imaging modalities by testing on a held-out pool of datasets. Use when the user wants to benchmark on nnDetection Medical Object Detection Benchmark, or asks about evaluating this task. Reports mAP@0.1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2106.00817
  bibtex_key: [REDACTED]
  confidence: high
---

# medical-object-detection-eval

> nnDetection: A Self-configuring Method for Medical Object Detection — Baumgartner et al. (2021) (arXiv:2106.00817, 2021)

## What this evaluates

Evaluates the ability of object detection models to localize and classify medical structures in 3D imaging data without manual hyperparameter tuning. It probes generalization across diverse anatomical regions and imaging modalities by testing on a held-out pool of datasets.

## Datasets

- **nnDetection Medical Object Detection Benchmark** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `mAP@0.1` **(primary)** — range: [0, 1]
  - Mean Average Precision computed at an Intersection over Union (IoU) threshold of 0.1. Calculated by averaging the precision-recall curves across all objects/classes.
- `CPM` — range: [0, 1]
  - Competition Performance Metric used specifically for the LUNA16 lung nodule detection task.
- `Sensitivity@0.3 FP` — range: [0, 1]
  - Sensitivity (recall) measured at a fixed false positive rate of 0.3 per scan, used for the ADAM aneurysm detection task.

## Input / output format

**Input**: 3D medical imaging volumes (primarily CT scans) with associated object annotations (center coordinates and radii or bounding boxes).

**Output**: Predicted object locations (center + radius or bounding boxes) with associated confidence scores.

## Scoring recipe

```python
def compute_map_at_01(preds, gts, iou_thresh=0.1):
    tp, fp = 0, 0
    for pred in preds:
        ious = [compute_iou(pred, gt) for gt in gts]
        if max(ious) >= iou_thresh:
            tp += 1
        else:
            fp += 1
    precisions = tp / (tp + fp + 1e-9)
    recalls = tp / (len(gts) + 1e-9)
    return interpolate_ap(recalls, precisions)
```

## Common pitfalls

- Using standard IoU thresholds (e.g., 0.5) instead of 0.1, which misaligns with clinical requirements for coarse localization in medical imaging.
- Evaluating on only a few small datasets, which causes high metric volatility and overfitting to specific anatomical regions or scanners.
- Comparing against segmentation-based detection substitutes without accounting for architectural differences, leading to unfair performance baselines.

## Evidence (verbatim from paper)

> Reflecting clinical relevance regarding coarse localisation on medical images and the absence of overlapping objects in 3D images, we report mean Average Precision (mAP) at an IoU threshold of 0.1 [9].

## Citation

```bibtex
@misc{baumgartner2021nndetection,
  title={nnDetection: A Self-configuring Method for Medical Object Detection},
  author={Baumgartner et al. (2021)},
  year={2021},
  note={arXiv:2106.00817}
}
```

- arXiv: 2106.00817