Gram Dti Eval

---
name: gram-dti-eval
description: Evaluates multimodal drug-target interaction prediction and zero-shot retrieval capabilities across multiple benchmark datasets and cold-start scenarios. Use when the user wants to benchmark on Activation, Yamanishi_08, Hetionet, Inhibition, or asks about evaluating this task. Reports AUPR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.21971
  bibtex_key: jiang2025gramdti
  confidence: high
---

# gram-dti-eval

> GRAM-DTI: adaptive multimodal representation learning for drug target interaction prediction — Feng Jiang et al. (2025) (arXiv:2509.21971, 2025)

## What this evaluates

Evaluates multimodal drug-target interaction prediction and zero-shot retrieval capabilities across multiple benchmark datasets and cold-start scenarios.

## Datasets

- **Activation** — total 1913; splits: train (-1), test (-1)
- **Yamanishi_08** — total 5127; splits: train (-1), test (-1)
- **Hetionet** — total 49942; splits: train (-1), test (-1)
- **Inhibition** — total 21055; splits: train (-1), test (-1)

## Metrics

- `AUPR` **(primary)** — range: [0, 1]
  - Area Under the Precision-Recall Curve. Computed by integrating precision over recall thresholds across all predicted probabilities.
- `AUROC` — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve. Computed by integrating the true positive rate over the false positive rate across all classification thresholds.
- `Recall@K` — range: [0, 1]
  - Proportion of relevant items retrieved within the top-K results. Calculated as the number of true positives in the top-K ranked candidates divided by the total number of true positives.

## Input / output format

**Input**: Multimodal representations of drug-target pairs: SMILES strings, text descriptions, HTA annotations, and protein sequences. For retrieval, single drug or protein queries.

**Output**: Binary interaction probability for DTI/MoA tasks; ranked candidate lists for retrieval tasks.

## Scoring recipe

```python
# DTI/MoA Prediction
preds = model.predict(pair)
auprc = average_precision_score(y_true, preds)
auroc = roc_auc_score(y_true, preds)

# Zero-shot Retrieval
scores = cosine_similarity(query_emb, candidate_embs)
top_k_indices = argsort(scores, descending=True)[:K]
recall_at_k = sum(1 for idx in top_k_indices if y_true[idx]) / len(y_true)
```

## Common pitfalls

- Negative samples are generated at a 1:10 ratio with positives, which differs from standard balanced or 1:1 sampling protocols.
- Cold-start splits enforce strict entity-level separation (drug or protein) to prevent data leakage, requiring careful fold construction.
- Cross-validation fold counts differ by task: 10-fold for DTI datasets and 5-fold for MoA datasets.

## Evidence (verbatim from paper)

> The performance is measured using standard metrics, including Recall@K (R@1, R@10, R@100), which indicate the proportion of relevant items retrieved within the top-K results.

## Citation

```bibtex
@misc{jiang2025gramdti,
  title={GRAM-DTI: adaptive multimodal representation learning for drug target interaction prediction},
  author={Feng Jiang et al. (2025)},
  year={2025},
  note={arXiv:2509.21971}
}
```

- arXiv: 2509.21971