Manifold Robustness Eval

---
name: manifold-robustness-eval
description: Evaluates the robustness of a dimensionality reduction pipeline (Isomap + Procrustes alignment + TDA clustering) against ambient noise, outliers, and hyperparameter variation. It tests whether the method can consistently recover a low-distortion 2D embedding of a contractible manifold or correctly detect topological failure on non-contractible data. Use when the user wants to benchmark on Swiss roll, Buckyball, or asks about evaluating this task. Reports Persistent homology features ($PH_1$, $PH_0$).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2408.01379
  bibtex_key: blumberg2024resampling
  confidence: high
---

# manifold-robustness-eval

> Resampling and averaging coordinates on data — Blumberg et al. (2024) (arXiv:2408.01379, 2024)

## What this evaluates

Evaluates the robustness of a dimensionality reduction pipeline (Isomap + Procrustes alignment + TDA clustering) against ambient noise, outliers, and hyperparameter variation. It tests whether the method can consistently recover a low-distortion 2D embedding of a contractible manifold or correctly detect topological failure on non-contractible data.

## Datasets

- **Swiss roll** — total 2000; splits: test (2000)
- **Buckyball** — total 60; splits: test (60)

## Metrics

- `Procrustes distances` — range: [0, inf)
  - Measures the optimal rotation/translation alignment cost between two embeddings. Used to cluster similar Isomap outputs.
- `Persistent homology features ($PH_1$, $PH_0$)` **(primary)** — range: topological invariants
  - Topological invariants computed via Ripser. Used to filter embeddings: good clusters have only small/no loops in $PH_1$ and a single connected component in $PH_0$. Coefficient field $\mathbb{F}_2$ is used.
- `Embedding quality (qualitative)` — range: qualitative
  - Visual and structural assessment of whether the averaged output successfully unrolls the manifold (e.g., Swiss roll) or correctly identifies failure (e.g., buckyball).

## Input / output format

**Input**: 3D point cloud coordinates (e.g., Swiss roll or Buckyball vertices), optionally corrupted with additive Gaussian noise or uniform outliers.

**Output**: 2D coordinate embedding (averaged, Procrustes-aligned representation of the underlying manifold).

## Scoring recipe

```python
embeddings = []
for subsample in subsamples:
    emb = isomap(subsample, k=neighborhood_size)
    embeddings.append(emb)
dist_matrix = procrustes_distance_matrix(embeddings)
ph_features = [persistent_homology(emb) for emb in embeddings]
good_cluster = [emb for emb, ph in zip(embeddings, ph_features) if ph['PH1_loops'] < threshold]
aligned_good = procrustes_align(good_cluster)
final_embedding = average(aligned_good)
```

## Common pitfalls

- Isomap is highly sensitive to the neighborhood size parameter; small radii cause disconnected components while large radii cause 'short-circuiting' and coiled embeddings.
- Persistent homology filtering requires careful choice of coefficient field (e.g., $\mathbb{F}_2$ vs $\mathbb{F}_3$) to correctly identify topological features of the embedding landscape.
- Procrustes alignment assumes embeddings are centered and in the same vector space; misalignment can artificially inflate distances or obscure true clusters.

## Evidence (verbatim from paper)

> The next step in the algorithm is to calculate Procrustes distances between these embeddings. ... The next step in the algorithm is to calculate the persistent homology of each embedding to identify the cluster consisting of the embeddings that have only very small (noise) loops in $PH_{1}$. This is precisely the cluster of the unrolled outputs. The final step is to align and average the embeddings in the unrolled cluster.

## Citation

```bibtex
@misc{blumberg2024resampling,
  title={Resampling and averaging coordinates on data},
  author={Blumberg et al. (2024)},
  year={2024},
  note={arXiv:2408.01379}
}
```

- arXiv: 2408.01379