Hescape Eval

---
name: hescape-eval
description: Evaluates cross-modal alignment between histology images and spatial gene expression profiles, and tests downstream capabilities in gene mutation classification and direct gene expression prediction from whole-slide images. Use when the user wants to benchmark on 5K, Multi, ImmOnc, Colon, Breast, Lung, or asks about evaluating this task. Reports Recall@5.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.01490
  bibtex_key: gindra2025hescape
  confidence: high
---

# hescape-eval

> A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics — Gindra et al. (2025) (arXiv:2508.01490, 2025)

## What this evaluates

Evaluates cross-modal alignment between histology images and spatial gene expression profiles, and tests downstream capabilities in gene mutation classification and direct gene expression prediction from whole-slide images.

## Datasets

- **5K** — total ?; splits: train (-1), test (-1); repo https://github.com/peng-lab/hescape
- **Multi** — total ?; splits: train (-1), test (-1); repo https://github.com/peng-lab/hescape
- **ImmOnc** — total ?; splits: train (-1), test (-1); repo https://github.com/peng-lab/hescape
- **Colon** — total ?; splits: train (-1), test (-1); repo https://github.com/peng-lab/hescape
- **Breast** — total ?; splits: train (-1), test (-1); repo https://github.com/peng-lab/hescape
- **Lung** — total ?; splits: train (-1), test (-1); repo https://github.com/peng-lab/hescape

## Metrics

- `Recall@5` **(primary)** — range: [0, 1]
  - Fraction of relevant gene expression profiles retrieved within the top 5 ranked predictions. Calculated as |retrieved_top5 ∩ relevant| / |relevant|.
- `F1 score` — range: [0, 1]
  - Harmonic mean of precision and recall for binary mutation status classification. Computed per biomarker and averaged or reported per class.
- `Pearson correlation coefficient (PCC)` — range: [-1, 1]
  - Linear correlation between predicted and actual continuous gene expression values across the test set.
- `mean squared error (MSE)` — range: [0, ∞)
  - Average of squared differences between predicted and actual gene expression values.

## Input / output format

**Input**: Histology image patches or whole-slide images paired with spatial gene expression profiles (retrieval); histology images alone (mutation/expression prediction).

**Output**: Ranked list of gene profiles (retrieval); binary mutation status labels (classification); continuous gene expression values (regression).

## Scoring recipe

```python
def recall_at_5(retrieved, relevant, k=5):
    return len(set(retrieved[:k]) & set(relevant)) / len(relevant)

def f1_score(y_true, y_pred):
    tp = sum((y_true == 1) & (y_pred == 1))
    fp = sum((y_true == 0) & (y_pred == 1))
    fn = sum((y_true == 1) & (y_pred == 0))
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    return 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0

def pcc_mse(y_true, y_pred):
    pcc = np.corrcoef(y_true, y_pred)[0, 1]
    mse = np.mean((y_true - y_pred) ** 2)
    return pcc, mse
```

## Common pitfalls

- Batch effects in gene expression data can skew cross-modal alignment and degrade downstream prediction performance.
- Contrastive pretraining may discard morphological features crucial for unimodal tasks like gene expression prediction.
- Performance gains are highly biomarker-specific and may not generalize across all genomic abnormalities.
- Limited spatial transcriptomics samples (e.g., only one Xenium sample per panel for LUAD) restrict pan-cancer generalization claims.

## Evidence (verbatim from paper)

> Our large-scale pretraining benchmark demonstrates that the gene encoder DRVI emerges as the key determinant of performance improvement across all tissue panels and both image-to-gene (I2G) and gene-to-image (G2I) metrics ([Tab. 2] and Supplementary[Tab. 7]-[Tab. 8]). Notably, DRVI paired with the image encoders Gigapath, H0mini, and UNI consistently achieves the top-performing Recall@5 Scores across datasets.

## Citation

```bibtex
@misc{gindra2025hescape,
  title={A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics},
  author={Gindra et al. (2025)},
  year={2025},
  note={arXiv:2508.01490}
}
```

- arXiv: 2508.01490