Softmol Molecular Generation Eval

---
name: softmol-molecular-generation-eval
description: Evaluates a diffusion-based molecular language model's ability to generate chemically valid, drug-like molecules and optimize them for specific protein targets. It probes distribution matching, structural diversity, and target-aware binding affinity prediction. Use when the user wants to benchmark on ZINC-Curated, SMILES, SAFE, or asks about evaluating this task. Reports Novel Top-hit 5% Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.21964
  bibtex_key: yang2026fromtokens
  confidence: high
---

# softmol-molecular-generation-eval

> From Tokens to Blocks: A Block-Diffusion Perspective on Molecular Generation — Qianwei Yang et al. (2026) (arXiv:2601.21964, 2026)

## What this evaluates

Evaluates a diffusion-based molecular language model's ability to generate chemically valid, drug-like molecules and optimize them for specific protein targets. It probes distribution matching, structural diversity, and target-aware binding affinity prediction.

## Datasets

- **ZINC-Curated** — total 427000000; splits: train (-1)
- **SMILES** — total 324000000; splits: train (-1)
- **SAFE** — total 322000000; splits: train (-1)

## Metrics

- `Novel Top-hit 5% Score` **(primary)** — range: kcal/mol
  - Mean docking score (DS) of the top 5% unique and novel generated hits. Hits must satisfy DS < median active DS, QED > 0.5, and SA < 5.0.
- `Quality` — range: percent
  - Proportion of unique, valid molecules meeting strict drug-likeness criteria: QED ≥ 0.6 and SA ≤ 4.
- `Docking-Filter` — range: percent
  - Proportion of generated molecules passing a pre-screening proxy for docking viability: QED > 0.5 and SA < 5.
- `Hit Ratio` — range: percent
  - Proportion of unique generated molecules that qualify as novel hits (DS < median active, QED > 0.5, SA < 5.0).
- `#Circles` — range: other
  - Number of distinct structural clusters among novel hits, measuring diversity and coverage of the explored chemical space (threshold set to 0.75).

## Input / output format

**Input**: Protein target identifier (e.g., parp1, fa7, 5ht1b, braf, jak2) and structural constraints for target-aware generation; or unconstrained prompts for de novo generation.

**Output**: Generated SMILES strings representing molecular structures.

## Scoring recipe

```python
def evaluate(generated_smiles, target=None):
    valid = [s for s in generated_smiles if is_valid(s)]
    unique = list(dict.fromkeys(valid))
    hits = [s for s in unique if qed(s) > 0.5 and sa(s) < 5.0]
    if target:
        ds_scores = [dock_score(s, target) for s in hits]
        top5 = sorted(ds_scores)[:max(1, len(ds_scores)//20)]
        return sum(top5)/len(top5) if top5 else float('inf')
    return {
        'uniqueness': len(unique)/len(generated_smiles),
        'hit_ratio': len(hits)/len(generated_smiles),
        'circles': count_clusters(hits, threshold=0.75)
    }
```

## Common pitfalls

- Failing to filter generated molecules for novelty (excluding training set members) before computing docking scores.
- Using incorrect QED/SA thresholds (paper specifies QED > 0.5, SA < 5.0 for hits, but QED ≥ 0.6, SA ≤ 4 for the Quality metric).
- Not averaging results over three independent random seeds as mandated by the protocol.
- Confusing the 'Unconstrained' SoftMol variant results with the main constrained model.

## Evidence (verbatim from paper)

> The primary evaluation metric is the Novel Top-hit 5% Score, defined as the mean docking score (DS) of the top 5% unique and novel generated hits.

## Citation

```bibtex
@misc{yang2026fromtokens,
  title={From Tokens to Blocks: A Block-Diffusion Perspective on Molecular Generation},
  author={Qianwei Yang et al. (2026)},
  year={2026},
  note={arXiv:2601.21964}
}
```

- arXiv: 2601.21964