Mol Exp Eval

---
name: mol-exp-eval
description: Evaluates a model's ability to explore chemical space and rediscover structurally diverse molecules with similar bioactivity against a specific target, rather than just optimizing for a single molecule. Use when the user wants to benchmark on MolExp, or asks about evaluating this task. Reports MolExpL.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2501.19153
  bibtex_key: thomas2025testtime
  confidence: high
---

# mol-exp-eval

> Test-Time Training Scaling Laws for Chemical Exploration in Drug Design — Thomas et al. (2025) (arXiv:2501.19153, 2025)

## What this evaluates

Evaluates a model's ability to explore chemical space and rediscover structurally diverse molecules with similar bioactivity against a specific target, rather than just optimizing for a single molecule.

## Datasets

- **MolExp** — total ?; splits: test (-1)

## Metrics

- `MolExpL` **(primary)** — range: [0, 1]
  - Product of the maximum Levenshtein-based string-edit similarity between each target molecule and the set of generated molecules: $\prod_{t \in T} \max_{m \in M} \text{sim}(m, t)$.
- `SEDiv` — range: [0, 1]
  - Sphere Exclusion Diversity measured at a sample size of 1,000, counting non-redundant molecules within a Tanimoto threshold of 0.65, normalized by sample size.

## Input / output format

**Input**: The model generates molecular SMILES strings. During evaluation, generation is guided by an RL oracle that provides a reward signal based on the maximum similarity to a set of target molecules.

**Output**: A set of generated molecular SMILES strings (up to a specified budget, e.g., 10,000 molecules per agent).

## Scoring recipe

```python
def compute_mol_exp_score(generated_smiles, target_smiles):
    score = 1.0
    for t in target_smiles:
        max_sim = max(levenshtein_similarity(m, t) for m in generated_smiles)
        score *= max_sim
    return score
```

## Common pitfalls

- The benchmark uses string-edit (Levenshtein) similarity for the primary MolExpL task, not fingerprint similarity, which can lead to different performance rankings.
- The metric is a product of similarities across all targets; failing to rediscover even a single target molecule drastically penalizes the score.
- Scaling the molecule budget for a single agent does not improve multi-target rediscovery; performance gains require scaling the number of independent RL agents.

## Evidence (verbatim from paper)

> Metric Performance is measured by calculating the product of the maximum similarity achieved to each target molecule $\prod_{t\in T}max(sim(m_i,m_t))$ . This metric focuses on the ability to generate similar molecules to all objectives. Molecular diversity is also measured by sphere exclusion diversity (SEDiv) at a sample size of 1,000 which provides a more accurate representation of chemical space coverage than internal diversity. $^{31,45}$ Note this is equivalent to the later published #Circles $^{45}$ metric at a Tanimoto threshold of 0.65 and normalized by the sample size.

## Citation

```bibtex
@misc{thomas2025testtime,
  title={Test-Time Training Scaling Laws for Chemical Exploration in Drug Design},
  author={Thomas et al. (2025)},
  year={2025},
  note={arXiv:2501.19153}
}
```

- arXiv: 2501.19153