BraTS PLGG Classification Eval

---
name: braTS-pLGG-classification-eval
description: Evaluates the effectiveness of synthetic 3D MRI tumor ROI generation for data augmentation by measuring downstream binary classification performance on imbalanced brain tumor subtypes. Use when the user wants to benchmark on BraTS 2019, SickKids pLGG, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2310.01251
  bibtex_key: zhou2023generating3d
  confidence: high
---

# braTS-pLGG-classification-eval

> Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks — Meng Zhou et al. (arXiv:2310.01251, 2023)

## What this evaluates

Evaluates the effectiveness of synthetic 3D MRI tumor ROI generation for data augmentation by measuring downstream binary classification performance on imbalanced brain tumor subtypes.

## Datasets

- **BraTS 2019** — total 335; splits: train (285), test (50)
- **SickKids pLGG** — total 214; splits: train (174), test (40)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve, measuring the model's ability to distinguish between classes across all classification thresholds.
- `Accuracy` — range: [0, 1]
  - Proportion of correctly classified instances out of the total test set instances.

## Input / output format

**Input**: 3D MRI tumor ROI volumes of size 128×128×128, normalized to [-1, 1].

**Output**: Binary classification label (HGG/LGG or BRAF Fusion/V600E).

## Scoring recipe

```python
def compute_metrics(predictions, labels):
    acc = np.mean(predictions == labels)
    auc = roc_auc_score(labels, predictions)
    return {'accuracy': acc, 'AUC': auc}
```

## Common pitfalls

- Test set is artificially balanced (25/25 or 20/20 per class) despite imbalanced training data, which may overestimate minority-class performance compared to real-world clinical distributions.
- Evaluation relies entirely on downstream classification metrics (AUC/Accuracy) rather than direct image quality measures (e.g., FID, SSIM), making it difficult to isolate generation fidelity from classifier robustness.

## Evidence (verbatim from paper)

> We randomly hold out 25 patients from both HGG and LGG as a standalone test set; these data are hidden from either training the 3D-VQGAN or the classifier. ... We use the 3D ResNet-50 model and trained for 50 epochs for all classification experiments. AUC and accuracy are computed on this test set.

## Citation

```bibtex
@misc{zhou2023generating3d,
  title={Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks},
  author={Meng Zhou et al.},
  year={2023},
  note={arXiv:2310.01251}
}
```

- arXiv: 2310.01251