Drug Target Interaction Eval

---
name: drug-target-interaction-eval
description: Evaluates computational models on predicting binary drug-target interactions using standardized bioactivity data. It probes the model's ability to learn molecular and protein representations and generalize across different data splits (lenient, cold-ligand, cold-target). Use when the user wants to benchmark on Curated DTI dataset, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2401.17174
  bibtex_key: golts2024drugtarget
  confidence: high
---

# drug-target-interaction-eval

> A large dataset curation and benchmark for drug target interaction — Golts et al. (2024) (arXiv:2401.17174, 2024)

## What this evaluates

Evaluates computational models on predicting binary drug-target interactions using standardized bioactivity data. It probes the model's ability to learn molecular and protein representations and generalize across different data splits (lenient, cold-ligand, cold-target).

## Datasets

- **Curated DTI dataset** — total 12644133; splits: Lenient (12644133), Cold-ligand (12318559), Cold-target (12318548)

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.
- `AUPR` — range: [0, 1]
  - Area under the Precision-Recall curve, measuring the trade-off between precision and recall across classification thresholds, particularly informative for imbalanced datasets.

## Input / output format

**Input**: Protein target sequence and ligand SMILES string.

**Output**: Probability of interaction (scalar between 0 and 1).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    from sklearn.metrics import roc_auc_score, average_precision_score
    auroc = roc_auc_score(y_true, y_pred)
    aupr = average_precision_score(y_true, y_pred)
    return {'AUROC': auroc, 'AUPR': aupr}
```

## Common pitfalls

- Dataset is highly imbalanced (~1.1-1.3% positive class), so AUROC can be misleading; AUPR is often more reliable for ranking performance.
- Results are split-dependent: 'Cold-ligand' and 'Cold-target' evaluate out-of-distribution generalization, while 'Lenient' evaluates in-distribution performance. Comparing across splits is invalid.
- Training is truncated to ~250k iterations (24h) on a single GPU, so reported metrics likely understate the model's full convergence potential.

## Evidence (verbatim from paper)

> Table 4. Benchmark results  

<table><tr><td>Split strategy</td><td>AUROC</td><td>AUPR</td></tr><tr><td>Lenient</td><td>0.843</td><td>0.145</td></tr><tr><td>Cold-ligand</td><td>0.833</td><td>0.136</td></tr><tr><td>Cold-target</td><td>0.671</td><td>0.029</td></tr></table>

## Citation

```bibtex
@misc{golts2024drugtarget,
  title={A large dataset curation and benchmark for drug target interaction},
  author={Golts et al. (2024)},
  year={2024},
  note={arXiv:2401.17174}
}
```

- arXiv: 2401.17174