Dti Binding Affinity Eval

---
name: dti-binding-affinity-eval
description: Evaluates a model's ability to predict continuous drug-target binding affinity and classify binary drug-target interactions. It probes geometry-aware representation learning, metric consistency, and generalization across diverse chemical-proteomic domains. Use when the user wants to benchmark on DTI-DG, BIOSNAP, BindingDB, DAVIS, or asks about evaluating this task. Reports PCC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.20693
  bibtex_key: refahi2025learning
  confidence: high
---

# dti-binding-affinity-eval

> Learning to Align Molecules and Proteins: A Geometry-Aware Approach to Binding Affinity — Refahi et al. (2025) (arXiv:2509.20693, 2025)

## What this evaluates

Evaluates a model's ability to predict continuous drug-target binding affinity and classify binary drug-target interactions. It probes geometry-aware representation learning, metric consistency, and generalization across diverse chemical-proteomic domains.

## Datasets

- **DTI-DG** — total ?; splits: test (-1)
- **BIOSNAP** — total ?; splits: test (-1)
- **BindingDB** — total ?; splits: test (-1)
- **DAVIS** — total ?; splits: test (-1)

## Metrics

- `PCC` **(primary)** — range: [-1, 1]
  - Pearson correlation coefficient between predicted and true binding affinity values. Higher values indicate better regression alignment.
- `AUPR` — range: [0, 1]
  - Area under the Precision-Recall Curve for binary drug-target interaction classification. Measures performance on imbalanced interaction data.
- `AUROC` — range: [0, 1]
  - Area under the Receiver Operating Characteristic Curve for binary drug-target interaction classification. Measures ranking quality of interaction probabilities.

## Input / output format

**Input**: Paired drug (e.g., SMILES/graph) and protein (e.g., sequence/embedding) representations.

**Output**: Continuous binding affinity score (regression) or binary interaction logit/probability (classification).

## Scoring recipe

```python
def compute_metrics(preds, golds, task='regression'):
    if task == 'regression':
        return pearsonr(preds, golds).statistic  # PCC
    else:
        preds_prob = sigmoid(preds)
        auprc = average_precision_score(golds, preds_prob)
        auroc = roc_auc_score(golds, preds_prob)
        return auprc, auroc
# Report mean ± s.e.m. over 5 random seeds.
```

## Common pitfalls

- Metrics are reported as mean ± standard error of the mean over 5 random seeds, not single-run values.
- DTI-DG is an out-of-domain generalization benchmark; standard in-distribution train/test splits do not apply.
- Task-specific metrics must be matched correctly: PCC for affinity regression, AUPR/AUROC for binary interaction classification.

## Evidence (verbatim from paper)

> Table[1] reports the mean PCC over the test domains when removing the FiLM conditioning layer or the triplet loss. ... Table[2] compares FIRM-DTI with strong baselines on three standard DTI datasets. Across BIOSNAP and BindingDB our model consistently achieves the highest or comparable AUPR and AUROC scores, showing that the FiLM-conditioned, geometry-aware representation generalizes well beyond affinity regression.

## Citation

```bibtex
@misc{refahi2025learning,
  title={Learning to Align Molecules and Proteins: A Geometry-Aware Approach to Binding Affinity},
  author={Refahi et al. (2025)},
  year={2025},
  note={arXiv:2509.20693}
}
```

- arXiv: 2509.20693