Molbench Eval

---
name: molbench-eval
description: Evaluates autonomous AI agents' ability to execute complex, multi-step drug discovery workflows, including molecular screening (property filtering, binding affinity comparison, docking) and molecular optimization (structural editing, physicochemical property improvement). Use when the user wants to benchmark on MolBench, or asks about evaluating this task. Reports optimization success rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.21937
  bibtex_key: [REDACTED]
  confidence: high
---

# molbench-eval

> MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization — Zhang et al. (2026) (arXiv:2604.21937, 2026)

## What this evaluates

Evaluates autonomous AI agents' ability to execute complex, multi-step drug discovery workflows, including molecular screening (property filtering, binding affinity comparison, docking) and molecular optimization (structural editing, physicochemical property improvement).

## Datasets

- **MolBench** — total ?; splits: MolBench-MS (112), MolBench-MO (39); repo https://github.com/InternScience/MolClaw

## Metrics

- `property filtering accuracy` — range: percent
  - Percentage of queries where the agent correctly applies threshold-based constraints to physicochemical descriptors.
- `property filtering F1 score` — range: percent
  - Harmonic mean of precision and recall for property filtering tasks.
- `binding affinity accuracy` — range: percent
  - Percentage of molecular pairs correctly ranked or classified by binding affinity using a validated multi-step docking pipeline.
- `docking hit count` — range: [0, 3]
  - Average number of valid docking hits returned in the top 3 predictions per target.
- `molecule editing accuracy` — range: percent
  - Percentage of tasks where the agent correctly applies specified chemical transformations (e.g., functional group replacement, scaffold hopping).
- `optimization delta` — range: other
  - Mean improvement score across three physicochemical properties (QED, LogP, LogS) after iterative structural modification and verification.
- `optimization success rate` **(primary)** — range: percent
  - Percentage of optimization tasks where all target physicochemical constraints are simultaneously satisfied after iterative refinement.

## Input / output format

**Input**: Molecular structures (SMILES/3D coordinates), target protein sequences/structures, physicochemical constraints, and optimization objectives.

**Output**: Generated molecular structures (SMILES/3D), computed property values, docking scores, and binary success indicators.

## Scoring recipe

```python
def evaluate_molbench(predictions, gold):
    scores = {}
    for task in ['prop_filter', 'bind_aff', 'edit']:
        scores[f'{task}_acc'] = sum(p == g for p, g in zip(predictions[task], gold[task])) / len(gold)
    scores['prop_filter_f1'] = f1_score(gold['prop_filter'], predictions['prop_filter'])
    scores['docking_hits'] = mean(len([s for s in pred[:3] if s > threshold]) for pred in predictions['docking'])
    scores['opt_delta'] = mean(pred['delta'] for pred in predictions['opt'])
    scores['opt_sr'] = sum(pred['success'] for pred in predictions['opt']) / len(gold)
    return scores
```

## Common pitfalls

- LLMs cannot reliably compute molecular descriptors through reasoning alone; tool access is the primary bottleneck for simple tasks.
- Vanilla agent frameworks lack structured multi-step pipelines, causing performance plateaus on complex tasks despite strong code generation ability.
- Optimization delta and success rate capture different aspects of iterative refinement; a high delta does not guarantee meeting all constraints.
- Statistical significance depends on task complexity; performance gains over near-ceiling baselines are often non-significant.

## Evidence (verbatim from paper)

> Across all seven metrics spanning the MolBench-MS and MolBench-MO benchmarks, MolClaw on Claude Code (MolClaw-CC) ranked first or tied for first on every metric, achieving sole best performance on four (binding affinity accuracy, docking hit count, molecule editing accuracy, and optimization delta) and tied best on three (property filtering accuracy and F1 score, optimization success rate)

## Citation

```bibtex
@misc{zhang2026molclaw,
  title={MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization},
  author={Zhang et al. (2026)},
  year={2026},
  note={arXiv:2604.21937}
}
```

- arXiv: 2604.21937