Cmu Mo Instruct Eval

---
name: cmu-mo-instruct-eval
description: Evaluates large language models' ability to generate valid, optimized molecular structures (SMILES) that satisfy multiple conflicting pharmacological and physicochemical property constraints. The benchmark probes the model's capacity for multi-objective reinforcement alignment, scaffold preservation, and strict adherence to property-wise improvement margins under both in-domain and out-of-distribution settings. Use when the user wants to benchmark on C-MuMOInstruct, or asks about evaluating this task. Reports Success Optimized Rate (Sor).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.23061
  bibtex_key: gao2026cmoral
  confidence: high
---

# cmu-mo-instruct-eval

> C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs — Gao et al. (2026) (arXiv:2604.23061, 2026)

## What this evaluates

Evaluates large language models' ability to generate valid, optimized molecular structures (SMILES) that satisfy multiple conflicting pharmacological and physicochemical property constraints. The benchmark probes the model's capacity for multi-objective reinforcement alignment, scaffold preservation, and strict adherence to property-wise improvement margins under both in-domain and out-of-distribution settings.

## Datasets

- **C-MuMOInstruct** — total ?; splits: train (100000), test (5000)

## Metrics

- `Success Optimized Rate (Sor)` **(primary)** — range: percent
  - Proportion of generated molecules that improve the targeted properties while maintaining other stability constraints.
- `Strict Success Optimized Rate (Ssor)` — range: percent
  - Percentage of candidates that improve sub-optimal properties, strictly maintain near-optimal constraints, and preserve the core scaffold.
- `Similarity (Sim)` — range: [0, 1]
  - Tanimoto similarity computed over Morgan fingerprints between generated candidates and initial molecules.
- `Relative Improvement (Ri)` — range: percent
  - Relative improvement calculated across all sub-optimal properties.

## Input / output format

**Input**: Initial molecule SMILES string and a set of target property constraints (improvement margins Δp and near-optimal thresholds Θp).

**Output**: Generated candidate molecule SMILES string.

## Scoring recipe

```python
def evaluate(predictions, gold, thresholds):
    valid_count = 0
    strict_valid_count = 0
    sims = []
    improvements = []
    for pred, init, props in zip(predictions, gold, thresholds):
        gen_props = compute_properties(pred)  # RDKit + ADMET-AI oracles
        init_props = compute_properties(init)
        if check_target_improvement(gen_props, init_props, props) and check_stability(gen_props, props):
            valid_count += 1
        if check_subopt_improvement(gen_props, init_props, props) and check_near_optimal(gen_props, props) and check_scaffold(pred, init):
            strict_valid_count += 1
        sims.append(tanimoto_similarity(morgan_fingerprint(pred), morgan_fingerprint(init)))
        improvements.append(relative_improvement(gen_props, init_props, props))
    return valid_count/len(predictions), strict_valid_count/len(predictions), sum(sims)/len(sims), sum(improvements)/len(improvements)
```

## Common pitfalls

- Must use the exact property thresholds (Δp, Θp) from Table 1 for each task; using default or generic thresholds will invalidate SOR/SSOR calculations.
- Property values are computed via external oracles (RDKit + ADMET-AI), not ground-truth experimental data, so oracle discrepancies directly affect scores.
- Generation uses beam search (width=20) followed by a specific candidate selection algorithm; skipping this step changes the evaluation distribution.

## Evidence (verbatim from paper)

> To comprehensively assess performance in highly constrained lead optimization, we employ four rigorous metrics: (1) Success Optimized Rate (Sor): the proportion of molecules that improve the targeted properties while maintaining other stability constraints; (2) Strict Success Optimized Rate (Ssor): percentage of candidates that improve sub-optimal properties and strictly maintain near-optimal ones while preserving the core scaffold; (3) Similarity (Sim): the Tanimoto similarity over Morgan fingerprints between generated candidates and initial molecules; (4) Relative Improvement (Ri): the relative improvement across all sub-optimal properties.

## Citation

```bibtex
@misc{gao2026cmoral,
  title={C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs},
  author={Gao et al. (2026)},
  year={2026},
  note={arXiv:2604.23061}
}
```

- arXiv: 2604.23061