Gmtkn55 Molecular Eval

---
name: gmtkn55-molecular-eval
description: Evaluates the accuracy of density functionals in predicting broad molecular properties, including atomization energies, barrier heights, and noncovalent interactions, relative to high-level quantum chemistry reference data. Use when the user wants to benchmark on GMTKN55 (specifically GMTKN53 subset), or asks about evaluating this task. Reports WTMAD1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2105.02696
  bibtex_key: [REDACTED]
  confidence: high
---

# gmtkn55-molecular-eval

> vdW-DF-ahcx: a range-separated van der Waals density functional hybrid — Shukla et al. (2021) (arXiv:2105.02696, 2021)

## What this evaluates

Evaluates the accuracy of density functionals in predicting broad molecular properties, including atomization energies, barrier heights, and noncovalent interactions, relative to high-level quantum chemistry reference data.

## Datasets

- **GMTKN55 (specifically GMTKN53 subset)** — total ?; splits: test (-1)

## Metrics

- `WTMAD1` **(primary)** — range: kcal/mol
  - Weighted Total Mean Absolute Deviation. Computes the MAD for each benchmark set, applies a weight based on the number of processes in the set, and averages them across all sets in the survey.
- `MAD` — range: kcal/mol
  - Mean Absolute Deviation. The average of the absolute differences between calculated and reference values for a given benchmark set.
- `TMAD` — range: kcal/mol
  - Total MAD. Averages the MAD values over multiple intermolecular or intramolecular noncovalent interaction benchmark sets.

## Input / output format

**Input**: Molecular geometries and reference energies/properties from the GMTKN55/GMTKN53 benchmark sets, or crystal structures for bulk metals/semiconductors/insulators.

**Output**: Calculated energies/properties (e.g., atomization energies, barrier heights, lattice constants, cohesive energies, bulk moduli) produced by the DFT functional.

## Scoring recipe

```python
def compute_wtmad1(calculated, reference, weights):
    abs_devs = [abs(c - r) for c, r in zip(calculated, reference)]
    mad = sum(abs_devs) / len(abs_devs)
    weighted_mad = mad * weights
    return sum(weighted_mad) / sum(weights)
```

## Common pitfalls

- The WATER27 and G21EA benchmark sets are explicitly excluded from the main GMTKN53 summary because they contain small negatively charged ions and radicals that are problematic for planewave pseudopotential codes.
- TMAD and WTMAD1 average errors over benchmark sets rather than individual molecular processes, which can mask performance on specific difficult subsets.
- Comparisons between planewave DFT codes and orbital-based codes must account for differences in pseudopotentials and basis sets to avoid unfair performance assessments.

## Evidence (verbatim from paper)

> We evaluate the weighted total mean absolute deviation measures (WTMAD1 and WTMAD2) that Grimme and co-workers introduced to allow a comparison also among the GMTKA55 groups.

## Citation

```bibtex
@misc{shukla2021vdwdfahcx,
  title={vdW-DF-ahcx: a range-separated van der Waals density functional hybrid},
  author={Shukla et al. (2021)},
  year={2021},
  note={arXiv:2105.02696}
}
```

- arXiv: 2105.02696