Ml4cfd Competition Eval

---
name: ml4cfd-competition-eval
description: Evaluates machine learning surrogates for 2D airfoil aerodynamics on prediction accuracy, computational speed-up, physical consistency, and out-of-distribution generalization. The benchmark compares learned models against a standard CFD solver (OpenFOAM) across in-distribution and novel geometric configurations. Use when the user wants to benchmark on ML4CFD Competition Dataset, or asks about evaluating this task. Reports Global Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.08516
  bibtex_key: [REDACTED]
  confidence: high
---

# ml4cfd-competition-eval

> NeurIPS 2024 ML4CFD Competition: Results and Retrospective Analysis — Yagoubi et al. (2024) (NeurIPS 2024 / arXiv:2506.08516, 2024)

## What this evaluates

Evaluates machine learning surrogates for 2D airfoil aerodynamics on prediction accuracy, computational speed-up, physical consistency, and out-of-distribution generalization. The benchmark compares learned models against a standard CFD solver (OpenFOAM) across in-distribution and novel geometric configurations.

## Datasets

- **ML4CFD Competition Dataset** — total ?; splits: in-distribution (-1), out-of-distribution (-1); repo https://github.com/IRT-SystemX/NeurIPS2024-ML4CFD-competition-Starting-Kit

## Metrics

- `Global Score` **(primary)** — range: percent
  - Weighted aggregation of ML performance (40%), physical consistency (30%), and OOD generalization (30%). ML score combines accuracy (75%) and speed-up (25%). OOD score combines OOD accuracy (42%), OOD physics (33%), and OOD speed-up (25%).
- `Accuracy` — range: other
  - Mean error across flow field variables ($\overline{u}_{x}$, $\overline{u}_{y}$, $\overline{p}$, $\overline{\nu}_{t}$, $\overline{p}_{s}$). Lower values indicate better performance.
- `Speed-up` — range: other
  - Ratio of CFD solver runtime to model inference runtime. Higher values indicate faster prediction.
- `Physical Criteria` — range: other
  - Evaluation of integral aerodynamic coefficients ($C_D$, $C_L$) and their correlation coefficients ($\rho_D$, $\rho_L$) against ground truth.

## Input / output format

**Input**: Airfoil geometry representation (e.g., mesh coordinates, latent embeddings, or surface parameters) and flow conditions (e.g., angle of attack, Reynolds number).

**Output**: Predicted flow field variables ($\overline{u}_{x}$, $\overline{u}_{y}$, $\overline{p}$, $\overline{\nu}_{t}$, $\overline{p}_{s}$) and integral aerodynamic coefficients ($C_D$, $C_L$).

## Scoring recipe

```python
# Weights from Table 1
ml_weight = 0.40
physics_weight = 0.30
ood_weight = 0.30

ml_score = (0.75 * accuracy_metric) + (0.25 * speedup_metric)
ood_score = (0.42 * ood_accuracy) + (0.33 * ood_physics) + (0.25 * ood_speedup)

global_score = (ml_weight * ml_score) + (physics_weight * physics_criteria_score) + (ood_weight * ood_score)
return global_score
```

## Common pitfalls

- Speed-up is heavily downweighted (25% in ML, 25% in OOD) compared to accuracy, so fast but inaccurate models rank poorly.
- Physical consistency ($C_D$, $C_L$, correlation coefficients) is evaluated separately from field accuracy, requiring models to preserve integral conservation laws.
- OOD evaluation uses distinct geometric configurations not seen during training, testing extrapolation rather than interpolation.

## Evidence (verbatim from paper)

> Rankings were determined based on a global score that aggregated performance across several key criteria, including machine learning (accuracy and speed-up), physics, and out-of-distribution (OOD) generalization.

## Citation

```bibtex
@misc{yagoubi2024ml4cfd,
  title={NeurIPS 2024 ML4CFD Competition: Results and Retrospective Analysis},
  author={Yagoubi et al. (2024)},
  year={2024},
  note={NeurIPS 2024 / arXiv:2506.08516}
}
```

- arXiv: 2506.08516