Soc Dgl Dti Eval

---
name: soc-dgl-dti-eval
description: This evaluation protocol assesses a model's capability to predict binary drug-target interactions (DTI) using graph-based representations. It specifically probes performance under both balanced and highly imbalanced data distributions, as well as generalization to unseen drugs or targets in cold-start scenarios. Use when the user wants to benchmark on KIBA, Davis, BindingDB, DrugBank, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.01405
  bibtex_key: zhao2025socdgl
  confidence: high
---

# soc-dgl-dti-eval

> SOC-DGL: Social Interaction Behavior Inspired Dual Graph Learning Framework for Drug-Target Interaction Identification — Zhao et al. (2025) (arXiv:2506.01405, 2025)

## What this evaluates

This evaluation protocol assesses a model's capability to predict binary drug-target interactions (DTI) using graph-based representations. It specifically probes performance under both balanced and highly imbalanced data distributions, as well as generalization to unseen drugs or targets in cold-start scenarios.

## Datasets

- **KIBA** — total ?; splits: 10-fold cross-validation (-1)
- **Davis** — total ?; splits: 10-fold cross-validation (-1)
- **BindingDB** — total ?; splits: 10-fold cross-validation (-1)
- **DrugBank** — total ?; splits: 10-fold cross-validation (-1)

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area under the receiver operating characteristic curve, measuring the trade-off between true positive rate and false positive rate across all classification thresholds.
- `AUPR` — range: [0, 1]
  - Area under the precision-recall curve, emphasizing performance on the positive class, particularly useful under class imbalance.
- `F1_score` — range: [0, 1]
  - Harmonic mean of precision and recall: 2 * (Precision * Recall) / (Precision + Recall).
- `ACC` — range: [0, 1]
  - Accuracy: (True Positives + True Negatives) / Total Samples.
- `Recall` — range: [0, 1]
  - True Positive Rate: True Positives / (True Positives + False Negatives).
- `Precision` — range: [0, 1]
  - Positive Predictive Value: True Positives / (True Positives + False Positives).
- `Specificity` — range: [0, 1]
  - True Negative Rate: True Negatives / (True Negatives + False Positives).

## Input / output format

**Input**: Drug and target molecular representations processed through dual graph learning modules (ADGL and EDGL) to form drug-target pairs for interaction prediction.

**Output**: Binary classification probability or discrete label (positive/negative) indicating whether a drug-target pair interacts.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred_proba, threshold=0.5):
    y_pred = (y_pred_proba >= threshold).astype(int)
    tp = np.sum((y_pred == 1) & (y_true == 1))
    tn = np.sum((y_pred == 0) & (y_true == 0))
    fp = np.sum((y_pred == 1) & (y_true == 0))
    fn = np.sum((y_pred == 0) & (y_true == 1))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    acc = (tp + tn) / len(y_true)
    specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
    auroc = roc_auc_score(y_true, y_pred_proba)
    auprc = average_precision_score(y_true, y_pred_proba)
    return {'AUROC': auroc, 'AUPR': auprc, 'F1_score': f1, 'ACC': acc, 'Recall': recall, 'Precision': precision, 'Specificity': specificity}
```

## Common pitfalls

- Evaluating on imbalanced datasets (1:10 positive:negative ratio) without using imbalance-aware loss functions or adjusting decision thresholds will severely skew Precision and Recall metrics.
- Cold-start experiments require strict isolation of unseen drugs or targets from the training graph; standard random splits will leak structural information and artificially inflate AUROC/AUPR.
- 10-fold cross-validation must be applied consistently across all baseline methods; reporting single-split results breaks comparability with the paper's statistical validation (paired t-tests + Fisher's Combined Probability Test).

## Evidence (verbatim from paper)

> To minimize data variability, 10-fold cross-validation was employed to evaluate model performance. During each fold, the test set was masked, and the performance is evaluated using metrics such as AUROC, AUPR, F1_score, ACC, Recall and Precision.

## Citation

```bibtex
@misc{zhao2025socdgl,
  title={SOC-DGL: Social Interaction Behavior Inspired Dual Graph Learning Framework for Drug-Target Interaction Identification},
  author={Zhao et al. (2025)},
  year={2025},
  note={arXiv:2506.01405}
}
```

- arXiv: 2506.01405