Embedding Clustering Eval

---
name: embedding-clustering-eval
description: This evaluation probes how well different embedding models capture underlying number-theoretic structures in numeric sequences. It measures the quality of the latent representation space by comparing clustering performance against ground-truth mathematical group labels versus unsupervised KMeans assignments. Use when the user wants to benchmark on number-theoretic-sequences, or asks about evaluating this task. Reports Silhouette Coefficient.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.21703
  bibtex_key: kabane2025evaluatingembedding
  confidence: high
---

# embedding-clustering-eval

> Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry — Kabane (2025) (arXiv:2511.21703, 2025)

## What this evaluates

This evaluation probes how well different embedding models capture underlying number-theoretic structures in numeric sequences. It measures the quality of the latent representation space by comparing clustering performance against ground-truth mathematical group labels versus unsupervised KMeans assignments.

## Datasets

- **number-theoretic-sequences** — total ?; splits: test (-1)

## Metrics

- `Silhouette Coefficient` **(primary)** — range: [-1, 1]
  - Quantifies how similar an object is to its own cluster compared to other clusters. Higher values indicate better-defined, more separated clusters.
- `Davies–Bouldin Index` — range: other
  - Computes the average similarity ratio of each cluster with its most similar one, based on scatter and separation measures. Lower values indicate better clustering quality.

## Input / output format

**Input**: Numeric sequences with associated number-theoretic group labels (for supervised evaluation) or unlabeled numeric sequences (for unsupervised KMeans evaluation).

**Output**: Embedding vectors for each sequence, followed by clustering assignments or direct metric computation against true labels.

## Scoring recipe

```python
embeddings = model.encode(sequences)
sil_sup = silhouette_score(embeddings, true_labels)
db_sup = davies_bouldin_score(embeddings, true_labels)
kmeans_labels = KMeans(n_clusters=k).fit_predict(embeddings)
sil_unsup = silhouette_score(embeddings, kmeans_labels)
db_unsup = davies_bouldin_score(embeddings, kmeans_labels)
return sil_sup, db_sup, sil_unsup, db_unsup
```

## Common pitfalls

- Negative Silhouette scores indicate poor global structure, but local regularities may still exist and be captured by KMeans.
- A higher KMeans Silhouette score than the true-label score suggests the embedding space clusters according to learned geometric priors rather than mathematical definitions.

## Evidence (verbatim from paper)

> Two standard metrics are employed: the Silhouette Coefficient (higher is better) and the Davies–Bouldin Index (lower is better).

## Citation

```bibtex
@misc{kabane2025evaluatingembedding,
  title={Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry},
  author={Kabane (2025)},
  year={2025},
  note={arXiv:2511.21703}
}
```

- arXiv: 2511.21703