Fs Mol Eval

---
name: fs-mol-eval
description: Few-shot molecular property prediction and regression on a large-scale benchmark with thousands of tasks. Probes generalization across diverse protein targets and varying support set sizes. Use when the user wants to benchmark on FS-Mol, or asks about evaluating this task. Reports ΔAUPRC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2205.02708
  bibtex_key: chen2022metalearning
  confidence: high
---

# fs-mol-eval

> Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction — Chen et al. (2022) (arXiv:2205.02708, 2022)

## What this evaluates

Few-shot molecular property prediction and regression on a large-scale benchmark with thousands of tasks. Probes generalization across diverse protein targets and varying support set sizes.

## Datasets

- **FS-Mol** — total 233786; splits: train (4938), val (40), test (157)

## Metrics

- `ΔAUPRC` **(primary)** — range: other
  - Change in area under the precision-recall curve between the model and a baseline. Used for binary classification tasks.
- `R²_os` — range: other
  - Predictive or out-of-sample coefficient of determination. Used for regression tasks.

## Input / output format

**Input**: Molecular compounds associated with specific protein targets. Support sets of varying sizes (16, 32, 64, 128, 256) for few-shot learning.

**Output**: Predicted activity values (binary or continuous IC50/EC50).

## Scoring recipe

```python
def score_fs_mol(predictions, labels, task_type):
    if task_type == 'classification':
        auprc = average_precision_score(labels, predictions)
        baseline_auprc = get_baseline_auprc()
        return auprc - baseline_auprc
    else:
        return r2_score(labels, predictions)
# Average over 10 stratified splits per task, then across tasks
```

## Common pitfalls

- Support sets are generally unbalanced for classification, reflecting real-world drug discovery distributions.
- Metrics are reported as changes (Δ) from a baseline for classification, not raw AUPRC.

## Evidence (verbatim from paper)

> The task-level metrics for binary classification and regression are ΔAUPRC (change in area under the precision-recall curve) and R²_os (predictive/out-of-sample coefficient of determination), respectively. ... averaged performance over ten different stratified support/query random splits of every test task is reported for each compared method.

## Citation

```bibtex
@misc{chen2022metalearning,
  title={Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction},
  author={Chen et al. (2022)},
  year={2022},
  note={arXiv:2205.02708}
}
```

- arXiv: 2205.02708