Marcel Eval

---
name: marcel-eval
description: Evaluates molecular property prediction using explicit conformer ensembles versus single-conformer or 1D/2D baselines, probing how 3D structural flexibility and ensemble encoding strategies impact regression accuracy. Use when the user wants to benchmark on MARCEL, or asks about evaluating this task. Reports Mean Absolute Error (MAE).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2310.00115
  bibtex_key: zhu2023marcel
  confidence: high
---

# marcel-eval

> Learning Over Molecular Conformer Ensembles: Datasets and Benchmarks — Zhu et al. (2023) (arXiv:2310.00115, 2023)

## What this evaluates

Evaluates molecular property prediction using explicit conformer ensembles versus single-conformer or 1D/2D baselines, probing how 3D structural flexibility and ensemble encoding strategies impact regression accuracy.

## Datasets

- **MARCEL** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/SXKDZ/MARCEL

## Metrics

- `Mean Absolute Error (MAE)` **(primary)** — range: other
  - MAE = (1/N) Σ |y_pred - y_true| over all test instances. Lower values indicate better performance.

## Input / output format

**Input**: Molecular structures represented as 1D sequences, 2D graphs, or 3D conformer ensembles (capped at 20 conformers per molecule).

**Output**: A single scalar value predicting the target molecular property.

## Scoring recipe

```python
def compute_mae(preds, gold):
    errors = [abs(p - g) for p, g in zip(preds, gold)]
    return sum(errors) / len(errors)
```

## Common pitfalls

- Uniform random sampling of conformers during training ignores Boltzmann weights, which can hurt performance on tasks sensitive to energy distributions.
- 3D models do not consistently outperform 1D/2D baselines on small datasets or properties insensitive to conformational changes.
- Capping encoded conformers to 20 per molecule is a stability trick that may alter learning dynamics and should be reported.

## Evidence (verbatim from paper)

> Each dataset is partitioned randomly into three subsets: 70% for training, 10% for validation, and 20% for test. ... The metric used is the Mean Absolute Error (MAE, ↓).

## Citation

```bibtex
@misc{zhu2023marcel,
  title={Learning Over Molecular Conformer Ensembles: Datasets and Benchmarks},
  author={Zhu et al. (2023)},
  year={2023},
  note={arXiv:2310.00115}
}
```

- arXiv: 2310.00115