Ssrs Retrosynthesis Eval

---
name: ssrs-retrosynthesis-eval
description: Evaluates single-step retrosynthesis capability by predicting reactant molecules from a given target product, testing both in-distribution chemical knowledge and out-of-distribution generalization. Use when the user wants to benchmark on USPTO-50K-test, URSA-expert-2026, or asks about evaluating this task. Reports Unique.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.03517
  bibtex_key: [REDACTED]
  confidence: high
---

# ssrs-retrosynthesis-eval

> MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery — Kuznetsov et al. (2026) (arXiv:2603.03517, 2026)

## What this evaluates

Evaluates single-step retrosynthesis capability by predicting reactant molecules from a given target product, testing both in-distribution chemical knowledge and out-of-distribution generalization.

## Datasets

- **USPTO-50K-test** — total ?; splits: test (-1)
- **URSA-expert-2026** — total ?; splits: test (-1)

## Metrics

- `Unique` **(primary)** — range: percent
  - Fraction of unique valid reactant sets among generated samples.
- `Max ChemCensor (CC)` — range: other
  - Per-target maximum ChemCensor score averaged over targets, measuring synthetic precedent plausibility.
- `Av. PT-Top-K CC` — range: other
  - Per-target average ChemCensor score over the top-K unique predictions.

## Input / output format

**Input**: Target molecule structure (SMILES) with a prompt requesting single-step retrosynthetic disconnection.

**Output**: Chain-of-thought reasoning in <think> tags, followed by an <answer> block containing predicted reactant SMILES.

## Scoring recipe

```python
preds = []
for rep in range(num_reps):
    out = model.generate(prompt(), target_mol)
    preds.append(parse_answer(out))
unique_preds = list(set(preds))
unique_frac = len(unique_preds) / len(preds)
max_cc = max(chemcensor_score(m) for m in unique_preds)
avg_cc_topk = mean(chemcensor_score(m) for m in unique_preds[:K])
```

## Common pitfalls

- ChemCensor scores require an external database of synthetic precedents; scores are not intrinsic to the model output.
- Top-K metrics depend on the value of K (e.g., @3, @5, @10) specified in the prompt.
- Reactant sets must be canonicalized and validated for chemical plausibility before scoring.

## Evidence (verbatim from paper)

> Columns report the following metrics. Unique: fraction of unique valid reactant sets among samples. Max: per-target maximum ChemCensor score averaged over targets. Av. PT-Top-K CC: per-target average ChemCensor score over top-K unique predictions.

## Citation

```bibtex
@misc{kuznetsov2026mmai,
  title={MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery},
  author={Kuznetsov et al. (2026)},
  year={2026},
  note={arXiv:2603.03517}
}
```

- arXiv: 2603.03517