Mmist Ccrcc Eval

---
name: mmist-ccrcc-eval
description: Evaluates multi-modal fusion and missing data imputation strategies for predicting 12-month survival in clear cell renal cell carcinoma (ccRCC) patients. It probes a model's ability to integrate heterogeneous clinical, genomic, and imaging data while handling severe modality missingness and class imbalance. Use when the user wants to benchmark on MMIST-ccRCC, or asks about evaluating this task. Reports BAcc.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2405.01658
  bibtex_key: mota2024mmistccrcc
  confidence: high
---

# mmist-ccrcc-eval

> MMIST-ccRCC: A Real World Medical Dataset for the Development of Multi-Modal Systems — Mota et al. (2024) (arXiv:2405.01658, 2024)

## What this evaluates

Evaluates multi-modal fusion and missing data imputation strategies for predicting 12-month survival in clear cell renal cell carcinoma (ccRCC) patients. It probes a model's ability to integrate heterogeneous clinical, genomic, and imaging data while handling severe modality missingness and class imbalance.

## Datasets

- **MMIST-ccRCC** — total 618; splits: train (-1), test (-1)

## Metrics

- `BAcc` **(primary)** — range: [0, 1]
  - Balanced Accuracy, defined as the average recall for both classes: BAcc = (Recall_positive + Recall_negative) / 2. Recall is calculated as TP / (TP + FN).

## Input / output format

**Input**: Per patient: multi-modal data including CT/MRI radiology scans, Whole Slide Images (WSI) for histopathology, and clinical/genomic features (ClinGen). Missing modalities are indicated by absence or placeholder vectors in the fusion pipeline.

**Output**: Binary classification label indicating whether the patient survived or died within 12 months.

## Scoring recipe

```python
def compute_bacc(y_true, y_pred):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    tn = np.sum((y_true == 0) & (y_pred == 0))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    recall_pos = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    recall_neg = tn / (tn + fp) if (tn + fp) > 0 else 0.0
    return (recall_pos + recall_neg) / 2.0
```

## Common pitfalls

- The dataset has extreme class imbalance (~88% survival), so standard accuracy will be misleadingly high and does not reflect minority class performance.
- Missing data is highly modality-specific (e.g., MRI only 8%, Genomics 74% missing), requiring explicit imputation or fusion strategies rather than simple deletion.
- The paper uses latent representations for fusion rather than raw images, so evaluators must replicate the feature extraction pipeline to match reported scores.

## Evidence (verbatim from paper)

> Given this extreme imbalance, we opted to evaluate the results in terms of BAcc, i.e., the average Recall for both classes.

## Citation

```bibtex
@misc{mota2024mmistccrcc,
  title={MMIST-ccRCC: A Real World Medical Dataset for the Development of Multi-Modal Systems},
  author={Mota et al. (2024)},
  year={2024},
  note={arXiv:2405.01658}
}
```

- arXiv: 2405.01658