Matq Eval

---
name: matq-eval
description: Evaluates the accuracy of a charge-equilibrated equivariant foundation potential in predicting atomic energies, forces, partial charges, and bulk mechanical/thermal properties across diverse crystalline, molecular, and ionic systems. Use when the user wants to benchmark on MatQ, Custom Model Systems (C10H2/C10H3+, Ag3+/−, Na8/9Cl8+, Au2-MgO(001)), or asks about evaluating this task. Reports RMSE, MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.07249
  bibtex_key: ko2025qet
  confidence: high
---

# matq-eval

> A Fast, Accurate, and Reactive Equivariant Foundation Potential — Ko et al. (2025) (arXiv:2511.07249, 2025)

## What this evaluates

Evaluates the accuracy of a charge-equilibrated equivariant foundation potential in predicting atomic energies, forces, partial charges, and bulk mechanical/thermal properties across diverse crystalline, molecular, and ionic systems.

## Datasets

- **MatQ** — total 120000; splits: test (-1)
- **Custom Model Systems (C10H2/C10H3+, Ag3+/−, Na8/9Cl8+, Au2-MgO(001))** — total ?; splits: test (-1)

## Metrics

- `RMSE` **(primary)** — range: other
  - Root mean squared error calculated as sqrt(mean((predicted - reference)^2)). Applied separately to energies, forces, and charges.
- `MAE` **(primary)** — range: other
  - Mean absolute error calculated as mean(|predicted - reference|). Applied to energies, forces, stresses, charges, bulk modulus, shear modulus, and heat capacity.

## Input / output format

**Input**: Atomic numbers (Z_i), atomic positions (R_i), and total system charge (Q_tot).

**Output**: Atomic energies (E_i), atomic forces, atomic partial charges (q_i), and total energy.

## Scoring recipe

```python
import numpy as np
def compute_rmse(pred, gold):
    return np.sqrt(np.mean((pred - gold) ** 2))
def compute_mae(pred, gold):
    return np.mean(np.abs(pred - gold))
# Apply per property (energy, force, charge, stress, modulus, heat capacity)
# Reference values are DFT-computed (PBE) or DDEC6 partitioned charges.
```

## Common pitfalls

- Comparing models trained on vastly different dataset sizes (e.g., MatQ ~120k vs OMat24 ~100M) without accounting for data scale effects on performance.
- Using traditional Ewald summation for long-range electrostatics in benchmarks, which contradicts the model's linear-scaling cutoff-based design and inflates computational cost.
- Confusing DDEC6 partitioned charges with other schemes like Hirshfeld or Bader, which yield different absolute values and should not be mixed in error calculations.

## Evidence (verbatim from paper)

> The test MAEs in energies, forces and stresses of the QET-MatQ FP are 33meV atoms−1, 108meV Å−1 and 0.452 GPa, respectively. The MAE of the charges of the QET-MatQ FP is approximately 36.4 me, which is only slightly larger than the MAEs for the custom QET MLIPs for the model systems in Table 1.

## Citation

```bibtex
@misc{ko2025qet,
  title={A Fast, Accurate, and Reactive Equivariant Foundation Potential},
  author={Ko et al. (2025)},
  year={2025},
  note={arXiv:2511.07249}
}
```

- arXiv: 2511.07249