Umd 3d Medical Seg Eval

---
name: umd-3d-medical-seg-eval
description: Evaluates the cross-modality generalization and robustness of 3D medical segmentation foundation models by testing their ability to segment 13 whole-body organs in functional (PET) versus structural (CT/MRI) imaging using intrinsically paired intra-subject scans. Use when the user wants to benchmark on UMD Benchmark, or asks about evaluating this task. Reports Dice Similarity Coefficient (DSC).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.07643
  bibtex_key: zhang2026umd
  confidence: medium
---

# umd-3d-medical-seg-eval

> Uncovering Modality Discrepancy and Generalization Illusion for General-Purpose 3D Medical Segmentation — Zhang et al. (2026) (arXiv:2602.07643, 2026)

## What this evaluates

Evaluates the cross-modality generalization and robustness of 3D medical segmentation foundation models by testing their ability to segment 13 whole-body organs in functional (PET) versus structural (CT/MRI) imaging using intrinsically paired intra-subject scans.

## Datasets

- **UMD Benchmark** — total 954; splits: test (954); repo https://github.com/YichiZhang98/UMD

## Metrics

- `Dice Similarity Coefficient (DSC)` **(primary)** — range: [0, 1]
  - Standard voxel-wise overlap metric for medical segmentation: DSC = 2 * |A ∩ B| / (|A| + |B|), where A and B are prediction and ground truth masks. Reported per organ and averaged across the 13 target organs.

## Input / output format

**Input**: 3D volumetric medical images (whole-body PET, CT, or MRI scans) acquired from the same subject during a single diagnostic session.

**Output**: Voxel-wise segmentation masks identifying 13 target organs: liver, left kidney, right kidney, brain, heart, spleen, aorta, lung, colon, urinary bladder, pancreas, esophagus, and stomach.

## Scoring recipe

```python
def compute_dice(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask) + np.sum(gt_mask)
    return 2.0 * intersection / union if union > 0 else 0.0

dsc_scores = [compute_dice(pred, gt) for pred, gt in zip(predictions, ground_truth)]
mean_dsc = np.mean(dsc_scores)
```

## Common pitfalls

- Assuming performance on structural modalities (CT/MRI) generalizes to functional modalities (PET), which the benchmark explicitly shows fails due to low-contrast metabolic signals.
- Using recompiled public datasets that risk data leakage, whereas this benchmark uses newly collected intra-subject paired scans to guarantee evaluation validity.
- Entangling task complexity with modality differences, obscuring true cross-modality robustness.

## Evidence (verbatim from paper)

> Voxel-wise annotations were provided for 13 diverse organs, including the liver, left kidney, right kidney, brain, heart, spleen, aorta, lung, colon, urinary bladder, pancreas, esophagus, and stomach. By providing ground-truth labels for both structural and functional imaging, our protocol enables a head-to-head comparison between different modalities.

## Citation

```bibtex
@misc{zhang2026umd,
  title={Uncovering Modality Discrepancy and Generalization Illusion for General-Purpose 3D Medical Segmentation},
  author={Zhang et al. (2026)},
  year={2026},
  note={arXiv:2602.07643}
}
```

- arXiv: 2602.07643