Dta Affinity Prediction Eval

---
name: dta-affinity-prediction-eval
description: Evaluates a model's ability to predict the binding affinity between small molecule drugs and protein targets. It probes regression accuracy, ranking consistency, and correlation strength on standardized drug-target interaction datasets. Use when the user wants to benchmark on Davis, KIBA, or asks about evaluating this task. Reports MSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2406.17697
  bibtex_key: xiao2024hgtdpdta
  confidence: high
---

# dta-affinity-prediction-eval

> HGTDP-DTA: Hybrid Graph-Transformer with Dynamic Prompt for Drug-Target Binding Affinity Prediction — Xi Xiao et al. (2024) (arXiv:2406.17697, 2024)

## What this evaluates

Evaluates a model's ability to predict the binding affinity between small molecule drugs and protein targets. It probes regression accuracy, ranking consistency, and correlation strength on standardized drug-target interaction datasets.

## Datasets

- **Davis** — total 30056; splits: train (25746), test (5010)
- **KIBA** — total 118254; splits: train (98545), test (19709)

## Metrics

- `MSE` **(primary)** — range: other
  - Mean Squared Error between predicted and ground-truth affinity values. Lower is better.
- `CI` — range: [0, 1]
  - Concordance Index measuring the probability that the model correctly ranks a randomly selected pair of drug-target interactions by affinity. Higher is better.
- `r2_m` — range: [0, 1]
  - Improved R-squared metric for external prediction, calculated as R^2 * (1 - sqrt(R^2_0 / R^2)), where R^2_0 is R^2 computed without an intercept. Higher is better.
- `Pearson` — range: [-1, 1]
  - Pearson correlation coefficient between predicted and actual binding affinities. Higher is better.

## Input / output format

**Input**: Drug SMILES strings and target protein sequences, processed into molecular graphs and sequence embeddings respectively.

**Output**: A single continuous floating-point value representing the predicted binding affinity (pKd for Davis, KIBA score for KIBA).

## Scoring recipe

```python
def compute_metrics(preds, golds):
    mse = np.mean((preds - golds) ** 2)
    pearson = np.corrcoef(preds, golds)[0, 1]
    pairs = list(itertools.combinations(range(len(preds)), 2))
    concordant = sum(1 for i, j in pairs if golds[i] != golds[j] and (preds[i]-preds[j])*(golds[i]-golds[j]) > 0)
    total = sum(1 for i, j in pairs if golds[i] != golds[j])
    ci = concordant / total if total > 0 else 0.0
    r2 = 1 - np.sum((preds - golds)**2) / np.sum((golds - np.mean(golds))**2)
    slope = np.sum(preds * golds) / np.sum(preds**2)
    r2_0 = 1 - np.sum((golds - slope * preds)**2) / np.sum((golds - np.mean(golds))**2)
    r2_m = r2 * (1 - np.sqrt(r2_0 / r2)) if r2 > 0 else 0.0
    return {'MSE': mse, 'CI': ci, 'r2_m': r2_m, 'Pearson': pearson}
```

## Common pitfalls

- The train/test splits are fixed per dataset; averaging over 5 random seeds applies only to model training initialization, not to the data split itself.
- The r2_m metric requires calculating R^2_0 without an intercept; using standard R^2 will yield incorrect values.
- CI calculation must correctly exclude pairs with tied ground-truth affinities, as they do not contribute to the concordant count.

## Evidence (verbatim from paper)

> Specifically, these metrics include the Mean Squared Error (MSE), Concordance Index (CI), $r^{2}_{m}$, and Pearson’s Correlation (Pearson).

## Citation

```bibtex
@misc{xiao2024hgtdpdta,
  title={HGTDP-DTA: Hybrid Graph-Transformer with Dynamic Prompt for Drug-Target Binding Affinity Prediction},
  author={Xi Xiao et al. (2024)},
  year={2024},
  note={arXiv:2406.17697}
}
```

- arXiv: 2406.17697