Molvision Benchmark Eval

---
name: molvision-benchmark-eval
description: Evaluates vision-language models on molecular property prediction by combining skeletal structure images with textual prompts. It probes the model's ability to perform binary classification, numerical regression, and textual description generation across diverse chemical properties. Use when the user wants to benchmark on BACE-V, BBBP-V, HIV-V, ClinTox-V, Tox21-V, ESOL-V, LD50-V, QM9-V, PCQM4Mv2-V, ChEBI-V, or asks about evaluating this task. Reports True/False accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.03283
  bibtex_key: [REDACTED]
  confidence: high
---

# molvision-benchmark-eval

> MolVision: Molecular Property Prediction with Vision Language Models — Adak et al. (2025) (arXiv:2507.03283, 2025)

## What this evaluates

Evaluates vision-language models on molecular property prediction by combining skeletal structure images with textual prompts. It probes the model's ability to perform binary classification, numerical regression, and textual description generation across diverse chemical properties.

## Datasets

- **BACE-V** — total ?; splits: train (1210), test (303)
- **BBBP-V** — total ?; splits: train (1640), test (410)
- **HIV-V** — total ?; splits: train (32902), test (8225)
- **ClinTox-V** — total ?; splits: train (1193), test (298)
- **Tox21-V** — total ?; splits: train (6265), test (1566)
- **ESOL-V** — total ?; splits: train (902), test (226)
- **LD50-V** — total ?; splits: train (5908), test (1477)
- **QM9-V** — total ?; splits: train (107000), test (27000)
- **PCQM4Mv2-V** — total ?; splits: train (3000000), test (700000)
- **ChEBI-V** — total ?; splits: train (32000), test (8000)

## Metrics

- `True/False accuracy` **(primary)** — range: [0, 1]
  - Fraction of correctly predicted binary labels (True/False) out of total test instances. Computed as correct predictions divided by total predictions.
- `RMSE` — range: other
  - Root Mean Squared Error between predicted and ground-truth numerical values for regression tasks.
- `Text similarity` — range: [0, 1]
  - Semantic or lexical overlap score between generated molecular descriptions and reference text.

## Input / output format

**Input**: Molecular skeletal structure image paired with a manually engineered text prompt containing SMILES or SELFIES representation.

**Output**: Textual response: 'True' or 'False' for classification tasks; a numerical value for regression tasks; or a descriptive sentence for description tasks.

## Scoring recipe

```python
def compute_metric(predictions, golds, task_type):
    if task_type == 'classification':
        correct = sum(1 for p, g in zip(predictions, golds) if p.strip().lower() == g.lower())
        return correct / len(predictions)
    elif task_type == 'regression':
        errors = [(float(p) - float(g))**2 for p, g in zip(predictions, golds)]
        return (sum(errors) / len(errors)) ** 0.5
    else:
        return text_similarity_score(predictions, golds)
```

## Common pitfalls

- Visual data alone is insufficient for accurate prediction; multimodal fusion is required for robust performance.
- Datasets use RDKit-generated skeletal images rather than experimental crystal structures, which may introduce representation bias.
- Manual prompt engineering limits direct comparison across different VLM architectures without standardized prompting.

## Evidence (verbatim from paper)

> For classification, we frame the task as a True/False question, where the model predicts whether a molecule inhibits a target property. For regression, the model generates a numerical value representing the target property and for description task, the model generates textual output.

## Citation

```bibtex
@misc{adak2025molvision,
  title={MolVision: Molecular Property Prediction with Vision Language Models},
  author={Adak et al. (2025)},
  year={2025},
  note={arXiv:2507.03283}
}
```

- arXiv: 2507.03283