Brats 2017 Eval

---
name: brats-2017-eval
description: Evaluates 3D brain tumor segmentation accuracy across three sub-regions (whole tumor, core, enhancing) and tests radiomics-based survival prediction performance on multi-modal MRI scans. Use when the user wants to benchmark on BraTS 2017, or asks about evaluating this task. Reports Dice score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1802.10508
  bibtex_key: isensee2018brats
  confidence: high
---

# brats-2017-eval

> Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge — Isensee et al. (2018) (arXiv:1802.10508, 2018)

## What this evaluates

Evaluates 3D brain tumor segmentation accuracy across three sub-regions (whole tumor, core, enhancing) and tests radiomics-based survival prediction performance on multi-modal MRI scans.

## Datasets

- **BraTS 2017** — total ?; splits: train (285), val (46), test (-1)

## Metrics

- `Dice score` **(primary)** — range: [0, 1]
  - Dice = 2 * |A ∩ B| / (|A| + |B|), measuring voxel-wise overlap between prediction and ground truth masks.
- `RMSE` — range: other
  - Root Mean Squared Error between predicted and actual survival times.
- `Spearman correlation` — range: [-1, 1]
  - Rank-based correlation coefficient between predicted and actual survival times.

## Input / output format

**Input**: 3D multi-modal MRI volumes (T1, T1ce, T2, FLAIR) with corresponding voxel-wise segmentation masks and patient survival labels.

**Output**: Voxel-wise segmentation masks for three tumor sub-regions and a predicted survival time/value.

## Scoring recipe

```python
def dice_score(pred_mask, gold_mask):
    intersection = np.sum(pred_mask * gold_mask)
    return 2.0 * intersection / (np.sum(pred_mask) + np.sum(gold_mask))

def rmse(pred_times, gold_times):
    return np.sqrt(np.mean((pred_times - gold_times) ** 2))
```

## Common pitfalls

- Cases with no enhancing tumor in the ground truth yield a Dice score of zero by definition, which can significantly lower the mean score.
- Manual ground truth annotations may contain errors, such as blood vessels being labeled as enhancing tumor, affecting segmentation metrics.
- Overfitting to the validation set is mitigated by limiting submissions, but test set performance may still drop due to difficult cases.

## Evidence (verbatim from paper)

> Quantitatively, we achieve Dice scores of 0.896, 0.797 and 0.732 for whole, core and enhancing, respectively, on the BraTS 2017 validation set.

## Citation

```bibtex
@misc{isensee2018brats,
  title={Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge},
  author={Isensee et al. (2018)},
  year={2018},
  note={arXiv:1802.10508}
}
```

- arXiv: 1802.10508