Mumo Instruct Eval

---
name: mumo-instruct-eval
description: Evaluates a model's ability to perform multi-objective molecular lead optimization by modifying a starting molecule to simultaneously improve multiple conflicting pharmacological properties while retaining structural similarity. Use when the user wants to benchmark on MuMO-Instruct, or asks about evaluating this task. Reports Success Rate (SR).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.03517
  bibtex_key: kuznetsov2026mmai
  confidence: high
---

# mumo-instruct-eval

> MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery — Kuznetsov et al. (2026) (arXiv:2603.03517, 2026)

## What this evaluates

Evaluates a model's ability to perform multi-objective molecular lead optimization by modifying a starting molecule to simultaneously improve multiple conflicting pharmacological properties while retaining structural similarity.

## Datasets

- **MuMO-Instruct** — total ?; splits: test (-1)

## Metrics

- `Success Rate (SR)` **(primary)** — range: percent
  - Percentage of generated molecules that satisfy all property optimization constraints.
- `Similarity (Sim)` — range: [0, 1]
  - Tanimoto similarity coefficient between the input and generated optimized molecule.
- `Relative Improvement (RI)` — range: other
  - Quantifies the magnitude of property enhancement achieved by the generated molecule relative to the baseline.

## Input / output format

**Input**: Molecular input (SMILES/structure) combined with a prompt assembled from randomly sampled paraphrased instruction templates.

**Output**: Chain-of-thought reasoning enclosed in <think> tags, followed by an <answer> block containing the generated molecule SMILES.

## Scoring recipe

```python
predictions = []
for rep in range(num_reps):
    prompt = sample_paraphrase()
    mol = augment_input(mol)
    out = model.generate(prompt, mol, stochastic=True)
    predictions.append(parse_answer_block(out))
final_pred = mode(predictions)
sr = sum(1 for m in final_pred if satisfies_constraints(m)) / len(final_pred)
sim = tanimoto_similarity(input_mol, final_pred)
ri = property_enhancement(input_mol, final_pred)
```

## Common pitfalls

- Models may output invalid SMILES strings that fail to parse or violate valence rules.
- Aggregation across stochastic decoding runs and prompt paraphrases is required; reporting a single greedy decode underestimates performance.
- Tanimoto similarity and property constraints must be computed on canonicalized molecular graphs.

## Evidence (verbatim from paper)

> Performance is measured using three metrics: Success Rate (SR), the percentage of generated molecules that satisfy all property optimization constraints; Similarity (Sim), the Tanimoto similarity between the input and generated optimized molecule; and Relative Improvement (RI), quantifying the magnitude of property enhancement.

## Citation

```bibtex
@misc{kuznetsov2026mmai,
  title={MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery},
  author={Kuznetsov et al. (2026)},
  year={2026},
  note={arXiv:2603.03517}
}
```

- arXiv: 2603.03517