Toximol Eval

---
name: toximol-eval
description: Evaluates whether Multimodal Large Language Models (MLLMs) can generate structurally valid, low-toxicity alternative molecules from toxic inputs while adhering to drug-likeness, synthetic feasibility, and structural similarity constraints. It probes the model's ability to perform structure-aware molecular editing and cross-modal scientific reasoning. Use when the user wants to benchmark on ToxiMol, or asks about evaluating this task. Reports Toxicity Repair Success Rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.10912
  bibtex_key: lin2025toximol
  confidence: high
---

# toximol-eval

> Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification? — Lin et al. (2025) (arXiv:2506.10912, 2025)

## What this evaluates

Evaluates whether Multimodal Large Language Models (MLLMs) can generate structurally valid, low-toxicity alternative molecules from toxic inputs while adhering to drug-likeness, synthetic feasibility, and structural similarity constraints. It probes the model's ability to perform structure-aware molecular editing and cross-modal scientific reasoning.

## Datasets

- **ToxiMol** — total 560; splits: test (560); repo https://github.com/DeepYoke/ToxiMol

## Metrics

- `Toxicity Repair Success Rate` **(primary)** — range: percent
  - Percentage of test samples where at least one of the generated candidate molecules passes the ToxiEval multi-dimensional evaluation chain. The chain requires the molecule to satisfy safety thresholds (S_safe), drug-likeness (QED), VEGA drug-likeness (V_ro5), synthetic accessibility (S_sas), and structural similarity (S_sim).

## Input / output format

**Input**: A toxic molecule represented by its SMILES string and 2D molecular image, accompanied by a natural language repair objective specifying the toxicity target to mitigate.

**Output**: Exactly 3 candidate molecule structures (SMILES) per input sample.

## Scoring recipe

```python
success_count = 0
for sample in test_set:
    candidates = model.generate(sample, k=3)
    sample_success = False
    for mol in candidates:
        if passes_toxieval(mol):  # checks S_safe, Q, V_ro5, S_sas, S_sim
            sample_success = True
            break
    if sample_success:
        success_count += 1
return (success_count / len(test_set)) * 100
```

## Common pitfalls

- High structural validity does not guarantee repair success; models often produce valid molecules that remain toxic or fail drug-likeness constraints.
- Increasing the number of generated candidates (k) yields diminishing returns, particularly for complex regression tasks like LD50 and SkinRxn.
- The QED (drug-likeness) metric acts as the strictest filter, frequently causing failures even when toxicity is successfully mitigated.

## Evidence (verbatim from paper)

> Table 1: Toxicity Repair Success Rates (%). ... A unified evaluation criterion is applied: a sample is successfully repaired if at least one candidate molecule passes the ToxiEval multi-dimensional evaluation chain. ... We report the per-task success rate and the overall success rate as the primary performance metrics.

## Citation

```bibtex
@misc{lin2025toximol,
  title={Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?},
  author={Lin et al. (2025)},
  year={2025},
  note={arXiv:2506.10912}
}
```

- arXiv: 2506.10912