Ukbob Medical Segmentation Eval

---
name: ukbob-medical-segmentation-eval
description: Evaluates 3D medical image segmentation models on multi-modal MRI and CT scans across abdominal, brain, and whole-body anatomical domains. Probes the model's ability to produce accurate organ and tumor masks while measuring both volumetric overlap and boundary precision under zero-shot and fine-tuning settings. Use when the user wants to benchmark on AMOS, BTCV, BRATS, UKBOB, or asks about evaluating this task. Reports Dice Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.06908
  bibtex_key: bourigault2025ukbob
  confidence: high
---

# ukbob-medical-segmentation-eval

> UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation — Bourigault et al. (2025) (arXiv:2504.06908, 2025)

## What this evaluates

Evaluates 3D medical image segmentation models on multi-modal MRI and CT scans across abdominal, brain, and whole-body anatomical domains. Probes the model's ability to produce accurate organ and tumor masks while measuring both volumetric overlap and boundary precision under zero-shot and fine-tuning settings.

## Datasets

- **AMOS** — total 100; splits: train (50), test (50)
- **BTCV** — total 50; splits: train (30), test (20)
- **BRATS** — total 5880; splits: test (-1)
- **UKBOB** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `Dice Score` **(primary)** — range: [0, 1]
  - Measures volumetric overlap between predicted and ground truth masks: 2 * |A ∩ B| / (|A| + |B|). Reported as mean across classes.
- `Hausdorff Distance` — range: other
  - Assesses boundary discrepancy by computing the maximum distance between any point on the predicted boundary and the closest point on the ground truth boundary. Reported as mean across classes.

## Input / output format

**Input**: 3D MRI or CT volumes, typically cropped to 96×96×96 voxels during training. Multi-modal intensity values are used as input channels.

**Output**: 3D segmentation masks (binary or multi-class) corresponding to target organs or tumors, matching the spatial dimensions of the input volume.

## Scoring recipe

```python
def compute_dice(pred, gt):
    intersection = np.sum(pred * gt)
    return 2.0 * intersection / (np.sum(pred) + np.sum(gt))

def compute_hausdorff(pred, gt):
    pred_boundary = find_boundaries(pred)
    gt_boundary = find_boundaries(gt)
    dists = cdist(pred_boundary, gt_boundary)
    return np.max(dists)

# Aggregate across classes and subjects
dice_scores = [compute_dice(p, g) for p, g in zip(predictions, ground_truth)]
hd_scores = [compute_hausdorff(p, g) for p, g in zip(predictions, ground_truth)]
mean_dice = np.mean(dice_scores)
mean_hd = np.mean(hd_scores)
```

## Common pitfalls

- Mean Dice Score can be heavily skewed by class imbalance; reporting per-class scores or weighted averages is often necessary for fair comparison.
- Hausdorff Distance is extremely sensitive to small boundary noise or outliers, which can inflate scores even when overall overlap is high.
- Datasets (AMOS, BTCV, BRATS) use different voxel spacings and coordinate conventions, requiring consistent resampling and orientation normalization before evaluation.

## Evidence (verbatim from paper)

> We evaluate our model using the Dice Score and the Hausdorff Distance Metric, which are widely used in medical image segmentation [[43], [34]]. The Dice Score measures the overlap between predicted and ground truth masks, while the Hausdorff Distance assesses the boundary discrepancy, providing a comprehensive evaluation of segmentation performance.

## Citation

```bibtex
@misc{bourigault2025ukbob,
  title={UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation},
  author={Bourigault et al. (2025)},
  year={2025},
  note={arXiv:2504.06908}
}
```

- arXiv: 2504.06908