Single Cell Clustering Eval

---
name: single-cell-clustering-eval
description: Evaluates clustering algorithms on single-cell RNA sequencing data to assess their ability to separate distinct cell types and capture transitional states. It combines quantitative clustering metrics with visual validation using marker gene expression profiles. Use when the user wants to benchmark on Breast cancer dataset, Embryo neurone dataset, or asks about evaluating this task. Reports misclustering rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.02061
  bibtex_key: ren2025bayesian
  confidence: high
---

# single-cell-clustering-eval

> A Bayesian approach to model uncertainty in single-cell genomic data — Ren et al. (2025) (arXiv:2508.02061, 2025)

## What this evaluates

Evaluates clustering algorithms on single-cell RNA sequencing data to assess their ability to separate distinct cell types and capture transitional states. It combines quantitative clustering metrics with visual validation using marker gene expression profiles.

## Datasets

- **Breast cancer dataset** — total ?; splits: test (-1)
- **Embryo neurone dataset** — total ?; splits: test (-1)

## Metrics

- `misclustering rate` **(primary)** — range: [0, 1]
  - A novel metric introduced to evaluate clustering performance without ground truth labels. It quantifies the proportion of cells assigned to clusters that are statistically indistinguishable or contain noise, validated via differential expression and SigClust tests.
- `NMI` — range: [0, 1]
  - Normalized Mutual Information, a standard clustering metric measuring the agreement between predicted clusters and reference labels, normalized to [0, 1].
- `ARI` — range: [0, 1]
  - Adjusted Rand Index, measuring the similarity between two clusterings while accounting for chance, ranging typically from -1 to 1 but reported here as positive values.
- `AUC` — range: [0, 1]
  - Area Under the ROC Curve used to measure the correspondence between clustering results and individual marker gene categories. Higher values indicate better discriminatory capacity for each marker.

## Input / output format

**Input**: Single-cell gene expression profiles (scRNA-seq data) reduced via UMAP-LE projection.

**Output**: Cluster assignments for each cell, with optional posterior probabilities of cluster membership (for VB-GMM).

## Scoring recipe

```python
def evaluate_clustering(predictions, labels=None, marker_genes=None):
    m_rate = compute_misclustering_rate(predictions)  # Novel unsupervised metric
    nmi = normalized_mutual_info(labels, predictions)
    ari = adjusted_rand_index(labels, predictions)
    auc_scores = [roc_auc_score(marker, cluster_probs) for marker in marker_genes]
    return {"misclustering_rate": m_rate, "NMI": nmi, "ARI": ari, "AUC": auc_scores}
```

## Common pitfalls

- Relying exclusively on quantitative metrics like NMI/ARI, which may appear low due to continuous biological states or imperfect ground truth, while ignoring visual marker gene validation.
- Assuming the misclustering rate follows standard accuracy conventions; it is specifically designed for unsupervised evaluation without ground truth labels.
- Ignoring the discrepancy between the K value optimal for quantitative metrics versus the K value optimal for visual marker separation.

## Evidence (verbatim from paper)

> Based on the results of the three metrics (misclustering rate, NMI and ARI), it indicates that the optimal numbers of clusters are  $K = 7$ ,  $K = 11$ , and  $K = 4$  for GMM and  $K = 5$ ,  $K = 11$  for VB-GMM (based on misclustering rate and ARI).

## Citation

```bibtex
@misc{ren2025bayesian,
  title={A Bayesian approach to model uncertainty in single-cell genomic data},
  author={Ren et al. (2025)},
  year={2025},
  note={arXiv:2508.02061}
}
```

- arXiv: 2508.02061