Msd Task1 Seg Eval

---
name: msd-task1-seg-eval
description: Evaluates the segmentation accuracy of a 3D U-Net model on brain tumor MRI volumes, and measures the computational efficiency of distributed hyperparameter tuning across multiple GPUs. Use when the user wants to benchmark on MSD Task 1, or asks about evaluating this task. Reports Dice score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2110.15884
  bibtex_key: berral2021distmis
  confidence: high
---

# msd-task1-seg-eval

> Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation — Berral et al. (2021) (arXiv:2110.15884, 2021)

## What this evaluates

Evaluates the segmentation accuracy of a 3D U-Net model on brain tumor MRI volumes, and measures the computational efficiency of distributed hyperparameter tuning across multiple GPUs.

## Datasets

- **MSD Task 1** — total 484; splits: train (-1), val (-1), test (-1); repo https://github.com/HiEST/DistMIS

## Metrics

- `Dice score` **(primary)** — range: [0, 1]
  - Computes the overlap between predicted and ground truth segmentation masks: 2 * |A ∩ B| / (|A| + |B|), where A and B are the sets of positive voxels in the prediction and ground truth respectively.

## Input / output format

**Input**: 4-channel 3D MRI volumes (T1w, T2w, T1gd, FLAIR) cropped to [240, 204, 152] in channel-first format.

**Output**: Binary 3D segmentation mask indicating whole tumor (positive) vs. background (negative).

## Scoring recipe

```python
def dice_score(pred, gt):
    pred = pred.flatten()
    gt = gt.flatten()
    intersection = sum(p * g for p, g in zip(pred, gt))
    return (2.0 * intersection) / (sum(pred) + sum(gt))
```

## Common pitfalls

- Full-volume 3D U-Net requires significant GPU memory, often necessitating batch size reduction or gradient accumulation.
- Learning rate must be scaled proportionally to the number of GPUs used in data parallelism to maintain convergence.
- Data parallelism incurs communication overhead when distributing across nodes, unlike experiment parallelism where runs are independent.

## Evidence (verbatim from paper)

> For the different experiments here performed, the evaluation on the validation and test sets provide a dice score of 0.89, which are the results of the state-of-art 3D U-Net model.

## Citation

```bibtex
@misc{berral2021distmis,
  title={Distributing Deep Learning Hyperparameter Tuning for 3D Medical Image Segmentation},
  author={Berral et al. (2021)},
  year={2021},
  note={arXiv:2110.15884}
}
```

- arXiv: 2110.15884