UMLIP High Temp Mof Eval

---
name: uMLIP-high-temp-mof-eval
description: Evaluates the accuracy of universal machine-learned interatomic potentials (uMLIPs) in predicting energy, forces, and stress tensors during high-temperature molecular dynamics simulations of metal-organic frameworks (MOFs), including their stability and thermal decomposition behavior. Use when the user wants to benchmark on High-Temperature MOF AIMD Benchmark, or asks about evaluating this task. Reports energy MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.25262
  bibtex_key: edwards2026benchmarking
  confidence: high
---

# uMLIP-high-temp-mof-eval

> Benchmarking Universal Machine-Learned Interatomic Potentials for High-Temperature Metal-Organic Framework Chemistry — Edwards et al. (2026) (arXiv:2604.25262, 2026)

## What this evaluates

Evaluates the accuracy of universal machine-learned interatomic potentials (uMLIPs) in predicting energy, forces, and stress tensors during high-temperature molecular dynamics simulations of metal-organic frameworks (MOFs), including their stability and thermal decomposition behavior.

## Datasets

- **High-Temperature MOF AIMD Benchmark** — total 9; splits: test (9)

## Metrics

- `energy MAE` **(primary)** — range: other
  - Mean absolute error between predicted and reference (AIMD) energy values, reported in meV/atom.
- `force MAE` — range: other
  - Mean absolute error between predicted and reference (AIMD) force vectors, reported in meV/Å.
- `stress MAE` — range: other
  - Mean absolute error between predicted and reference (AIMD) stress tensors, reported in MPa.

## Input / output format

**Input**: Atomic configurations of nine zinc- and zirconium-based MOFs at 300, 1000, and 2000 K, extracted from 40 ps ab initio molecular dynamics (AIMD) trajectories.

**Output**: Predicted scalar energy (meV/atom), force vectors (meV/Å), and stress tensors (MPa) for each configuration.

## Scoring recipe

```python
def compute_mae(predictions, references):
    return np.mean(np.abs(predictions - references))
# Applied independently to energy, force, and stress components.
# Final metric = mean across all 9 MOFs and 3 temperatures.
```

## Common pitfalls

- Static validation metrics (MAE) significantly underestimate errors that accumulate during long-timescale dynamic simulations (generative errors).
- Force and stress predictions are consistently less accurate than energy predictions, even near equilibrium, which can mislead evaluations relying solely on energy MAE.
- Model rankings on this benchmark correlate with MATBench, but performance is highly system-dependent, with some MOFs showing systematically higher errors regardless of the model.

## Evidence (verbatim from paper)

> The error was broken down into energy, force and stress components and averaged across all MOFs and temperatures (Table 1). ORB-v3 and fairchem OMAT show low energy mean absolute error (MAE) at 3.59 and 3.21 meV atom-1, respectively, while MACE models and fairchem ODAC23 show errors exceeding 10 meV atom-1.

## Citation

```bibtex
@misc{edwards2026benchmarking,
  title={Benchmarking Universal Machine-Learned Interatomic Potentials for High-Temperature Metal-Organic Framework Chemistry},
  author={Edwards et al. (2026)},
  year={2026},
  note={arXiv:2604.25262}
}
```

- arXiv: 2604.25262