Task01 Braintumour Eval

---
name: task01-braintumour-eval
description: Evaluates a model's ability to perform 3D semantic segmentation on multi-parametric MRI scans of brain tumours. It probes the algorithm's capacity to distinguish and delineate multiple tumour sub-regions (edema, enhancing, non-enhancing) under high clinical variability in scanner hardware and acquisition protocols. Use when the user wants to benchmark on Task01_BrainTumour, or asks about evaluating this task. Reports semantic segmentation accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1902.09063
  bibtex_key: simpson2019msd
  confidence: high
---

# task01-braintumour-eval

> A large annotated medical image dataset for the development and evaluation of segmentation algorithms — Amber L. Simpson et al. (arXiv:1902.09063, 2019)

## What this evaluates

Evaluates a model's ability to perform 3D semantic segmentation on multi-parametric MRI scans of brain tumours. It probes the algorithm's capacity to distinguish and delineate multiple tumour sub-regions (edema, enhancing, non-enhancing) under high clinical variability in scanner hardware and acquisition protocols.

## Datasets

- **Task01_BrainTumour** — total 750; splits: train (-1), test (-1)

## Metrics

- `semantic segmentation accuracy` **(primary)** — range: [0, 1]
  - Standard volumetric overlap metric, typically Dice coefficient: 2|A∩B|/(|A|+|B|), where A and B are the predicted and ground-truth segmentation masks. Measures voxel-wise overlap between predicted and expert-annotated tumour sub-regions.

## Input / output format

**Input**: 3D multi-parametric MRI volumes (T1, T1-Gd, T2, T2-FLAIR) in NIfTI format, co-registered to SRI24 atlas space, resampled to 1mm³ isotropic voxels, and skull-stripped.

**Output**: 3D segmentation mask in NIfTI format with discrete labels for each tumour sub-region (edema, enhancing, non-enhancing).

## Scoring recipe

```python
def compute_dice(pred_mask, gold_mask):
    intersection = np.sum(pred_mask * gold_mask)
    union = np.sum(pred_mask) + np.sum(gold_mask)
    return 2.0 * intersection / union if union > 0 else 0.0
# Compute per-sub-region, then average across sub-regions and subjects.
```

## Common pitfalls

- Images are rotated 180 degrees in the axial plane to correct misformatted headers; models must assume RAS coordinate frame.
- Multi-modal input required (4 sequences per scan); failing to concatenate or process all modalities degrades performance.
- Highly variable acquisition protocols across 19 institutions and 1T-3T scanners; models trained on single-site data often fail to generalize.

## Evidence (verbatim from paper)

> The datasets were chosen based on availability and appropriateness for semantic segmentation algorithm development. Gold standard annotations for all tumour sub-regions in all scans were approved by expert board-certified neuradiologists.

## Citation

```bibtex
@misc{simpson2019msd,
  title={A large annotated medical image dataset for the development and evaluation of segmentation algorithms},
  author={Amber L. Simpson et al.},
  year={2019},
  note={arXiv:1902.09063}
}
```

- arXiv: 1902.09063