Brats2023 Segmentation Eval

---
name: brats2023-segmentation-eval
description: Evaluates the zero-shot and fine-tuned performance of promptable and non-promptable 3D medical image segmentation models on brain tumor MRI data. It probes how prompt type (points vs. bounding boxes) and prompt accuracy affect segmentation quality compared to a strong unprompted baseline. Use when the user wants to benchmark on BraTS 2023 Adult Glioma, BraTS 2023 Pediatrics, or asks about evaluating this task. Reports Dice score (DSC).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.20786
  bibtex_key: ludwig2025ai
  confidence: high
---

# brats2023-segmentation-eval

> AI-Driven MRI-based Brain Tumour Segmentation Benchmarking — Ludwig et al. (2025) (arXiv:2506.20786, 2025)

## What this evaluates

Evaluates the zero-shot and fine-tuned performance of promptable and non-promptable 3D medical image segmentation models on brain tumor MRI data. It probes how prompt type (points vs. bounding boxes) and prompt accuracy affect segmentation quality compared to a strong unprompted baseline.

## Datasets

- **BraTS 2023 Adult Glioma** — total ?; splits: train (-1)
- **BraTS 2023 Pediatrics** — total ?; splits: train (-1), test (-1)

## Metrics

- `Dice score (DSC)` **(primary)** — range: [0, 1]
  - Dice Similarity Coefficient measuring overlap between predicted and ground truth segmentation masks: 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Reported as a value between 0 and 1.

## Input / output format

**Input**: 3D MRI brain volumes with optional prompts (1, 5, or 10 points; or high/medium/low quality bounding boxes) indicating tumor location.

**Output**: 3D binary segmentation mask of the brain tumor region.

## Scoring recipe

```python
def compute_dice(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask) + np.sum(gt_mask)
    if union == 0:
        return 1.0
    return 2.0 * intersection / union
```

## Common pitfalls

- Zero-shot evaluation on the adult glioma dataset suffers from data leakage, inflating scores.
- Prompt quality is subjective; 'high/medium/low' box accuracy and point count significantly impact results, making zero-shot prompting impractical without precise ground-truth prompts.
- Video segmentation mode consistently underperforms image mode for SAM 2, but is often reported without distinction.

## Evidence (verbatim from paper)

> Table I shows the average Dice scores obtained by all models across all applicable prompting methods. Of these results, MedSAM, SAM-Med-3D and nnU-net all benefit from data leakage to varying degrees. nnU-Net has the highest Dice score across all models and prompting styles with a score of 0.958 despite being unprompted.

## Citation

```bibtex
@misc{ludwig2025ai,
  title={AI-Driven MRI-based Brain Tumour Segmentation Benchmarking},
  author={Ludwig et al. (2025)},
  year={2025},
  note={arXiv:2506.20786}
}
```

- arXiv: 2506.20786