Medical Seg Uncertainty Eval

---
name: medical-seg-uncertainty-eval
description: This evaluation protocol assesses the segmentation accuracy and robustness of 3D medical imaging models under varying data quality and training strategies. It specifically probes how aleatoric uncertainty quantification can guide data filtering and dynamic loss weighting to improve performance across diverse anatomical structures and imaging modalities. Use when the user wants to benchmark on LiTS, TotalSegmentator, WORD, FeTA 2022, KiTS23, or asks about evaluating this task. Reports Dice score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.10963
  bibtex_key: li2026delving
  confidence: high
---

# medical-seg-uncertainty-eval

> Delving Aleatoric Uncertainty in Medical Image Segmentation via Vision Foundation Models — Li et al. (2026) (arXiv:2604.10963, 2026)

## What this evaluates

This evaluation protocol assesses the segmentation accuracy and robustness of 3D medical imaging models under varying data quality and training strategies. It specifically probes how aleatoric uncertainty quantification can guide data filtering and dynamic loss weighting to improve performance across diverse anatomical structures and imaging modalities.

## Datasets

- **LiTS** — total 201; splits: train (131), test (70)
- **TotalSegmentator** — total 1204; splits: train (1082), val (57), test (65)
- **WORD** — total 120; splits: (unstated)
- **FeTA 2022** — total 120; splits: (unstated)
- **KiTS23** — total 489; splits: (unstated)

## Metrics

- `Dice score` **(primary)** — range: percent
  - Computed as 2 * |intersection| / (|prediction| + |ground_truth|) across all classes, reported as a percentage.
- `mIoU` — range: percent
  - Mean Intersection over Union across all semantic classes, calculated as the average of IoU per class, reported as a percentage.

## Input / output format

**Input**: 3D medical image volumes (CT or MRI) preprocessed with nnU-Net normalization and resampling, provided as 96x96x96 patches during training. Ground truth voxel-wise segmentation masks are provided for supervision.

**Output**: Voxel-wise predicted segmentation masks (class labels) for each input volume.

## Scoring recipe

```python
def compute_metrics(pred_mask, gt_mask, num_classes):
    dice_scores = []
    ious = []
    for c in range(num_classes):
        pred_c = (pred_mask == c)
        gt_c = (gt_mask == c)
        intersection = np.sum(pred_c & gt_c)
        union = np.sum(pred_c | gt_c)
        if union == 0:
            dice_scores.append(1.0)
            ious.append(1.0)
        else:
            dice_scores.append(2 * intersection / (np.sum(pred_c) + np.sum(gt_c)))
            ious.append(intersection / union)
    return np.mean(dice_scores) * 100, np.mean(ious) * 100
```

## Common pitfalls

- Reporting training set metrics instead of test set metrics for the best epoch, which the paper explicitly warns against.
- Assuming a uniform 20% test split applies to all datasets, whereas LiTS and TotalSegmentator use fixed, published splits.
- Confusing the data retention percentage (90% vs 95%) with the uncertainty threshold; the paper filters out the top 5% or 10% most uncertain samples based on AUV.

## Evidence (verbatim from paper)

> We adopt the Dice score and the mean Intersection over Union (mIoU) as evaluation metrics. To fairness, we report the inference results of the model on the test set rather than the training results for the best epoch.

## Citation

```bibtex
@misc{li2026delving,
  title={Delving Aleatoric Uncertainty in Medical Image Segmentation via Vision Foundation Models},
  author={Li et al. (2026)},
  year={2026},
  note={arXiv:2604.10963}
}
```

- arXiv: 2604.10963