Brats Oasis1 Uncertainty Eval

---
name: brats-oasis1-uncertainty-eval
description: Evaluates a deep learning model's ability to perform multi-region brain MRI segmentation (tumors and healthy structures) while simultaneously predicting per-voxel uncertainty. It probes the model's segmentation accuracy across anatomical regions and its calibration of confidence estimates against actual voxel-wise errors. Use when the user wants to benchmark on BraTS, OASIS-1, or asks about evaluating this task. Reports DSC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.12801
  bibtex_key: zhou2025enhancing
  confidence: high
---

# brats-oasis1-uncertainty-eval

> Enhancing Neuro-Oncology Through Self-Assessing Deep Learning Models for Brain Tumor Unified Model for MRI Segmentation — Zhou (2025) (arXiv:2511.12801, 2025)

## What this evaluates

Evaluates a deep learning model's ability to perform multi-region brain MRI segmentation (tumors and healthy structures) while simultaneously predicting per-voxel uncertainty. It probes the model's segmentation accuracy across anatomical regions and its calibration of confidence estimates against actual voxel-wise errors.

## Datasets

- **BraTS** — total ?; splits: test (-1)
- **OASIS-1** — total ?; splits: test (-1)

## Metrics

- `DSC` **(primary)** — range: [0, 1]
  - Dice Similarity Coefficient measuring overlap between predicted and ground truth segmentation masks. Calculated as 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Reported per region (e.g., whole tumor, tumor core, enhancing tumor, whole brain).
- `RMSD` — range: other
  - Root Mean Square Deviation between the model's predicted uncertainty map and the true voxel-wise segmentation error (absolute difference between prediction and ground truth). Lower values indicate better localized uncertainty calibration.
- `CORR` — range: other
  - Pearson correlation coefficient between the predicted uncertainty map and the true voxel-wise segmentation error. Higher values indicate better rank-order calibration of confidence.

## Input / output format

**Input**: 3D MRI volumes (single modality for unified models, four modalities for cancer-only models).

**Output**: Multi-label segmentation mask (54 labels including tumor and healthy brain structures) and a continuous per-voxel uncertainty map.

## Scoring recipe

```python
def compute_metrics(pred_mask, gt_mask, uncertainty_map):
    # DSC per region
    intersection = np.sum((pred_mask == region_id) & (gt_mask == region_id))
    union = np.sum(pred_mask == region_id) + np.sum(gt_mask == region_id)
    dsc = 2 * intersection / union
    # True error map
    true_error = np.abs(pred_mask - gt_mask)
    # RMSD
    rmsd = np.sqrt(np.mean((uncertainty_map - true_error) ** 2))
    # CORR
    corr = np.corrcoef(uncertainty_map.flatten(), true_error.flatten())[0, 1]
    return dsc, rmsd, corr
```

## Common pitfalls

- DSC is highly sensitive to class imbalance; small structures like the hippocampus or enhancing tumor may show disproportionately low scores despite clinically acceptable performance.
- Uncertainty metrics (RMSD, CORR) require a well-defined 'true error' map; the paper uses voxel-wise absolute difference between prediction and ground truth, which may not align with other uncertainty evaluation protocols that use ensemble variance or Monte Carlo dropout.
- The unified model was trained on only one MRI modality for compatibility with OASIS-1, which limits direct performance comparison with 4-modality baselines like CM1/CM2.

## Evidence (verbatim from paper)

> Table[1] reports the cancer segmentation DSC for the different brain tumor regions and the uncertainty performance metric for the CM and UM models. ... Metrics demonstrate that the UM1 model excels at uncertainty prediction, as its Pearson correlation coefficient of 0.75 represents state-of-the-art performance compared to other studies (below). The CM1 model achieved a correlation coefficient of 0.634 for uncertainty prediction, which is lower than UM1 but still remarkable considering the model had to segment 54 different region labels. Interestingly, the voxel-wise uncertainty RMSD metrics reveal a different trend: UM1 achieved 0.047 while CM1 achieved 0.032, with CM1 actually performing better. In fact, the RMSD metric reported for CM1 is the lowest among the studies compared in Table[2]. The RMSD value provides a direct voxel-wise evaluation between predicted uncertainty and real error and indicates that our models, particularly CM1, performed better than existing statistics-based methods in localized uncertainty prediction.

## Citation

```bibtex
@misc{zhou2025enhancing,
  title={Enhancing Neuro-Oncology Through Self-Assessing Deep Learning Models for Brain Tumor Unified Model for MRI Segmentation},
  author={Zhou (2025)},
  year={2025},
  note={arXiv:2511.12801}
}
```

- arXiv: 2511.12801