Ppb Affinity Eval

---
name: ppb-affinity-eval
description: Evaluates protein language model architectures for predicting binding affinity in multi-chain protein-protein complexes. It probes how well different architectural designs capture inter-chain interactions compared to simple sequence or embedding concatenation. Use when the user wants to benchmark on PPB-Affinity, or asks about evaluating this task. Reports Spearman ρ.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.20036
  bibtex_key: alsamkary2025beyond
  confidence: high
---

# ppb-affinity-eval

> Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction — Alsamkary et al. (2025) (arXiv:2505.20036, 2025)

## What this evaluates

Evaluates protein language model architectures for predicting binding affinity in multi-chain protein-protein complexes. It probes how well different architectural designs capture inter-chain interactions compared to simple sequence or embedding concatenation.

## Datasets

- **PPB-Affinity** — total ?; splits: val (-1), test (-1); repo https://github.com/Proteinea/ppiseq

## Metrics

- `Spearman ρ` **(primary)** — range: [-1, 1]
  - Spearman rank correlation coefficient between predicted and true binding affinities.
- `Pearson r` — range: [-1, 1]
  - Pearson linear correlation coefficient between predicted and true binding affinities.
- `RMSE (pKd)` — range: [0, ∞)
  - Root mean squared error between predicted and true pKd values, calculated as sqrt(mean((y_pred - y_true)^2)).

## Input / output format

**Input**: Paired protein sequences representing ligand and receptor chains for multi-chain complexes.

**Output**: A single continuous scalar value representing the predicted binding affinity (pKd).

## Scoring recipe

```python
import numpy as np
from scipy.stats import spearmanr, pearsonr

def compute_metrics(y_true, y_pred):
    spearman_rho, _ = spearmanr(y_true, y_pred)
    pearson_r, _ = pearsonr(y_true, y_pred)
    rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
    return spearman_rho, pearson_r, rmse
```

## Common pitfalls

- Data leakage can occur if sequence identity between splits exceeds 30%; the protocol enforces a strict ≤30% identity split.
- Simple concatenation (SC) of all chains into one sequence with a single EOS token obscures ligand-receptor boundaries, diluting inter-chain interaction signals.
- Full fine-tuning of certain PLMs (e.g., ESM3-SM-Open) can cause training instability with exploding hidden states, requiring careful learning rate and normalization adjustments.

## Evidence (verbatim from paper)

> Metrics (mean ± standard deviation, 3 seeds): Spearman  $
ho$ , Pearson r, and RMSE  $(pK_d)$ . PAD: Pooled attention addition; HP: Hierarchical pooling

## Citation

```bibtex
@misc{alsamkary2025beyond,
  title={Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction},
  author={Alsamkary et al. (2025)},
  year={2025},
  note={arXiv:2505.20036}
}
```

- arXiv: 2505.20036