Davis Complete Eval

---
name: davis-complete-eval
description: Evaluates protein-ligand binding affinity prediction models on a modification-aware dataset, testing their ability to generalize across different train-test splits (new ligands, new proteins, modifications) and assessing robustness to wild-type overfitting and few-shot fine-tuning. Use when the user wants to benchmark on DAVIS-complete, or asks about evaluating this task. Reports Rp.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.00708
  bibtex_key: wu2025daviscomplete
  confidence: high
---

# davis-complete-eval

> Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset — Ming-Hsiu Wu et al. (2025) (arXiv:2512.00708, 2025)

## What this evaluates

Evaluates protein-ligand binding affinity prediction models on a modification-aware dataset, testing their ability to generalize across different train-test splits (new ligands, new proteins, modifications) and assessing robustness to wild-type overfitting and few-shot fine-tuning.

## Datasets

- **DAVIS-complete** — total 4032; splits: train (-1), test (-1), wild-type subset (-1), modification subset (-1); repo https://github.com/ZhiGroup/DAVIS-complete

## Metrics

- `Rp` **(primary)** — range: other
  - Pearson correlation coefficient between predicted and true $pK_d$ values.
- `MSE` — range: other
  - Mean Squared Error between predicted and true $pK_d$ values.
- `C-index` — range: other
  - Concordance index measuring the probability that predicted and true affinity rankings are consistent.

## Input / output format

**Input**: Protein sequence/structure and ligand structure (SMILES or 3D coordinates depending on model type).

**Output**: Predicted binding affinity as a continuous $pK_d$ value.

## Scoring recipe

```python
def compute_metrics(pred, true):
    mse = np.mean((pred - true) ** 2)
    rp = np.corrcoef(pred, true)[0, 1]
    c_index = concordance_index(true, pred)
    return mse, rp, c_index
```

## Common pitfalls

- Models may overfit to wild-type predictions, yielding high correlation on modified pairs by simply echoing seen wild-type values rather than learning modification effects.
- Performance heavily depends on whether proteins or ligands are seen during training; new-ligand splits are generally harder than new-protein splits.
- Capped vs uncapped affinity values in test subsets significantly impact results, with capped values masking poor generalization.

## Evidence (verbatim from paper)

> MSE, $R_p$, and C-index are computed between predicted and true $pK_d$ values.

## Citation

```bibtex
@misc{wu2025daviscomplete,
  title={Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset},
  author={Ming-Hsiu Wu et al. (2025)},
  year={2025},
  note={arXiv:2512.00708}
}
```

- arXiv: 2512.00708