Peptide Protein Interaction Eval

---
name: peptide-protein-interaction-eval
description: Evaluates a model's ability to predict whether a given peptide-protein pair interacts (binary classification) and to localize binding residues on both the peptide and protein sequences. It also assesses the model's capacity to generate target-specific peptide sequences that improve structural binding affinity over native templates. Use when the user wants to benchmark on Test167, LEADS-PEP, Test251, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.18467
  bibtex_key: [REDACTED]
  confidence: high
---

# peptide-protein-interaction-eval

> An Integrated Deep-Learning Framework for Peptide-Protein Interaction Prediction and Target-Conditioned Peptide Generation with ConGA-PePPI and TC-PepGen — Tang et al. (2026) (arXiv:2604.18467, 2026)

## What this evaluates

Evaluates a model's ability to predict whether a given peptide-protein pair interacts (binary classification) and to localize binding residues on both the peptide and protein sequences. It also assesses the model's capacity to generate target-specific peptide sequences that improve structural binding affinity over native templates.

## Datasets

- **Test167** — total ?; splits: test (-1)
- **LEADS-PEP** — total ?; splits: test (-1)
- **Test251** — total ?; splits: test (-1)

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.
- `AUPR` — range: [0, 1]
  - Area under the Precision-Recall curve, summarizing performance across thresholds, particularly sensitive to class imbalance.
- `MCC` — range: [-1, 1]
  - Matthews Correlation Coefficient, a balanced measure for binary classification that accounts for true and false positives and negatives.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall.
- `ipTM hit rate` — range: percent
  - Percentage of generated peptide sequences whose predicted interface TM-score (ipTM) from Chai-1, ESMFold, or AlphaFold 3 exceeds that of the native test peptide.

## Input / output format

**Input**: Paired peptide and protein amino acid sequences.

**Output**: Binary interaction label (interacting vs. non-interacting) or interaction probability score; for generation, a peptide amino acid sequence.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred_proba):
    y_pred = (y_pred_proba >= 0.5).astype(int)
    return {
        'AUROC': roc_auc_score(y_true, y_pred_proba),
        'AUPR': average_precision_score(y_true, y_pred_proba),
        'F1': f1_score(y_true, y_pred),
        'MCC': matthews_corrcoef(y_true, y_pred),
        'ACC': accuracy_score(y_true, y_pred),
        'Precision': precision_score(y_true, y_pred),
        'Recall': recall_score(y_true, y_pred)
    }
```

## Common pitfalls

- The evaluation uses temporally separated and low-homology test sets (e.g., Test167 removes pairs with >80% sequence similarity to training data) to prevent data leakage, which must be strictly enforced.
- Binding-site prediction suffers from severe class imbalance due to sparse interface residues, making AUPR a more reliable metric than accuracy or AUROC for the protein side.
- Generation evaluation relies on predicted structural scores (ipTM) rather than experimental validation, so high ipTM hit rates indicate structural plausibility but not necessarily biological activity.

## Evidence (verbatim from paper)

> We used Test167, a temporally separated low-homology set constructed after removing pairs with sequence similarity above 80% to the training or validation data, and we further evaluated the model on LEADS-PEP and Test251. Across these three benchmarks, ConGA-PepPI remains competitive relative to representative sequence-based and structure-based baselines, and it achieves an AUROC of 0.936 on LEADS-PEP.

## Citation

```bibtex
@misc{tang2026congapippi,
  title={An Integrated Deep-Learning Framework for Peptide-Protein Interaction Prediction and Target-Conditioned Peptide Generation with ConGA-PePPI and TC-PepGen},
  author={Tang et al. (2026)},
  year={2026},
  note={arXiv:2604.18467}
}
```

- arXiv: 2604.18467