Glioma Idh Prediction Eval

---
name: glioma-idh-prediction-eval
description: This benchmark evaluates a model's ability to predict glioma IDH mutation status (mutant vs. wild-type) by integrating multi-modal MRI data, including anatomical sequences, tumor geometry, and reconstructed brain networks. It probes the model's capacity for cross-modal feature alignment and patient-level binary classification under data-scarce conditions. Use when the user wants to benchmark on TCIA & In-house Glioma Cohort, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2203.10852
  bibtex_key: wei2022multimodal
  confidence: medium
---

# glioma-idh-prediction-eval

> Multi-modal learning for predicting the genotype of glioma — Wei et al. (2022) (arXiv:2203.10852, 2022)

## What this evaluates

This benchmark evaluates a model's ability to predict glioma IDH mutation status (mutant vs. wild-type) by integrating multi-modal MRI data, including anatomical sequences, tumor geometry, and reconstructed brain networks. It probes the model's capacity for cross-modal feature alignment and patient-level binary classification under data-scarce conditions.

## Datasets

- **TCIA & In-house Glioma Cohort** — total 524; splits: train (270), test (117), self-supervised_pretrain (20)

## Metrics

- `Accuracy` **(primary)** — range: [0, 1]
  - Standard binary classification accuracy: the proportion of correctly predicted IDH mutation labels (mutant vs. wild-type) out of the total test instances.

## Input / output format

**Input**: Per patient: four co-registered MRI sequences (pre-contrast T1, post-contrast T1, T2, T2-FLAIR) resampled to 2mm and cropped to 120x120x120; tumor point cloud coordinates; and brain network node/edge features derived from anatomical and diffusion MRI.

**Output**: Binary label indicating IDH mutation status (1 for mutant, 0 for wild-type).

## Scoring recipe

```python
def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)
```

## Common pitfalls

- The self-supervised pre-training set (20 patients) is drawn from the in-house cohort, but the exact overlap with the main train/test split is not explicitly detailed, risking data leakage.
- The 7:3 train/test split is applied to a subset of 387 patients, but the total cohort size sums to 524; the discrepancy in patient counts across sections may cause confusion during reproduction.
- Segmentation quality is validated using a DICE score, but this is a preprocessing quality check, not the primary evaluation metric for genotype prediction.

## Evidence (verbatim from paper)

> Finally, 407 of 424 patients are included with 105 IDH mutants and 302 IDH wild-types. ... The testing set includes 117 patients from the publicly available TCIA website. ... We apply binary cross-entropy loss for patient classification.

## Citation

```bibtex
@misc{wei2022multimodal,
  title={Multi-modal learning for predicting the genotype of glioma},
  author={Wei et al. (2022)},
  year={2022},
  note={arXiv:2203.10852}
}
```

- arXiv: 2203.10852