Tdc Adme Pk Eval

---
name: tdc-adme-pk-eval
description: Assesses pharmacological property prediction across ADME, PK, and toxicity tasks, including regression, classification, and correlation-based evaluation. Use when the user wants to benchmark on TDC Benchmark, or asks about evaluating this task. Reports AUROC / AUPRC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.03517
  bibtex_key: kuznetsov2026mmai
  confidence: high
---

# tdc-adme-pk-eval

> MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery — Kuznetsov et al. (2026) (arXiv:2603.03517, 2026)

## What this evaluates

Assesses pharmacological property prediction across ADME, PK, and toxicity tasks, including regression, classification, and correlation-based evaluation.

## Datasets

- **TDC Benchmark** — total ?; splits: test (-1)

## Metrics

- `MAE` — range: other
  - Mean Absolute Error for regression tasks (e.g., Caco2 permeability, HIA).
- `Spearman's Correlation` — range: [-1, 1]
  - Spearman rank correlation coefficient for pharmacokinetics tasks.
- `AUROC / AUPRC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic / Precision-Recall Curve for classification tasks (e.g., BBB penetration, hERG inhibition).

## Input / output format

**Input**: Molecular input (SMILES/structure) with task-specific instruction prompt.

**Output**: Chain-of-thought reasoning in <think> tags, followed by an <answer> block containing a numeric value or class label.

## Scoring recipe

```python
preds = []
for rep in range(num_reps):
    out = model.generate(prompt(), mol)
    preds.append(parse_answer(out))
final = median(preds) if regression else mode(preds)
if task_type == 'regression':
    mae = mean(abs(p - g) for p, g in zip(final, gold))
elif task_type == 'pk':
    spearman = rank_correlation(final, gold)
else:
    auroc = roc_auc_score(gold, class_probs)
    auprc = pr_auc_score(gold, class_probs)
```

## Common pitfalls

- TDC tasks have strict data splits and preprocessing pipelines; deviating from the standard protocol invalidates comparisons.
- Class probabilities must be extracted from first-token logprobs via softmax, not from the final discrete prediction.
- Spearman correlation is used for PK tasks instead of Pearson or RMSE, which is a common point of confusion.

## Evidence (verbatim from paper)

> Following the standard TDC protocol, we utilize task-specific metrics including Mean Absolute Error (MAE) for regression tasks, Spearman’s Correlation for pharmacokinetics, and AUROC or AUPRC for classification tasks

## Citation

```bibtex
@misc{kuznetsov2026mmai,
  title={MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery},
  author={Kuznetsov et al. (2026)},
  year={2026},
  note={arXiv:2603.03517}
}
```

- arXiv: 2603.03517