Gromov Wasserstein Similarity

---
name: gromov-wasserstein-similarity
description: Evaluates how well Gromov-Wasserstein distance captures functional similarity between neural network layer representations, enabling the identification of structural transitions and latent sub-networks across varying dimensionalities without task-specific supervision. Use when the user has predictions and gold and needs to compute Gromov-Wasserstein distance.
metadata:
  skill_kind: metric
  source_arxiv: 2410.16484
  bibtex_key: gao2024subnetworks
  confidence: medium
---

# gromov-wasserstein-similarity

> Identifying Sub-networks in Neural Networks via Functionally Similar Representations — Tian Gao et al. (2024) (arXiv:2410.16484, 2024)

## What this evaluates

Evaluates how well Gromov-Wasserstein distance captures functional similarity between neural network layer representations, enabling the identification of structural transitions and latent sub-networks across varying dimensionalities without task-specific supervision.

## Datasets

- **f_mod3** — total 3481; splits: train (-1), val (-1)

## Metrics

- `Gromov-Wasserstein distance` **(primary)** — range: [0, ∞)
  - Computes the optimal transport cost between two probability distributions (layer activations) without requiring them to share the same space or dimensionality. It minimizes the discrepancy between pairwise distances in the source and target distributions.

## Input / output format

**Input**: Intermediate activation tensors from specified neural network layers (e.g., transformer blocks, ResNet-9 modules).

**Output**: Scalar GW distance values between pairs of layer representations, indicating functional similarity.

## Scoring recipe

```python
def compute_gw_distance(X, Y, cost_X, cost_Y, reg=0.1):
    # X, Y: activation matrices (n_samples, d1), (m_samples, d2)
    # cost_X, cost_Y: pairwise distance matrices for X and Y
    # Uses standard GW optimal transport solver
    return gw_solver(X, Y, cost_X, cost_Y, reg)
```

## Common pitfalls

- GW distance computation scales cubically with sample size, requiring subsampling for large batches.
- Results are sensitive to the choice of cost matrix (typically Euclidean) and regularization parameter.
- Layer activations must be properly normalized or centered before distance computation to avoid scale bias.

## Evidence (verbatim from paper)

> The paper proposes a task-agnostic, automated method to identify functionally distinct sub-networks in neural networks using Gromov-Wasserstein (GW) distance, which enables direct comparison of intermediate layer representations across different dimensionalities and distributions.

## Citation

```bibtex
@misc{gao2024subnetworks,
  title={Identifying Sub-networks in Neural Networks via Functionally Similar Representations},
  author={Tian Gao et al. (2024)},
  year={2024},
  note={arXiv:2410.16484}
}
```

- arXiv: 2410.16484