Bubbleml Eval

---
name: bubbleml-eval
description: This benchmark evaluates machine learning models on two distinct scientific tasks using the BubbleML dataset: predicting optical flow for bubble dynamics and solving multiphysics PDEs for temperature and velocity field propagation. It probes a model's ability to capture non-rigid object motion, sharp physical interfaces, and long-horizon temporal dynamics in phase-change simulations. Use when the user wants to benchmark on BubbleML, or asks about evaluating this task. Reports end-point error (EPE).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2307.14623
  bibtex_key: hassan2023bubbleml
  confidence: high
---

# bubbleml-eval

> BubbleML: A Multi-Physics Dataset and Benchmarks for Machine Learning — Sheikh Md Shakeel Hassan et al. (2023) (arXiv:2307.14623, 2023)

## What this evaluates

This benchmark evaluates machine learning models on two distinct scientific tasks using the BubbleML dataset: predicting optical flow for bubble dynamics and solving multiphysics PDEs for temperature and velocity field propagation. It probes a model's ability to capture non-rigid object motion, sharp physical interfaces, and long-horizon temporal dynamics in phase-change simulations.

## Datasets

- **BubbleML** — total 2500; splits: train (2000), val (500); repo https://github.com/HPCForge/BubbleML

## Metrics

- `end-point error (EPE)` **(primary)** — range: other
  - Average L2 distance between predicted and ground truth flow vectors across all pixels. Lower is better.
- `RMSE` — range: other
  - Root Mean Squared Error between predicted and ground truth physical fields (temperature or velocity). Lower is better.
- `IRMSE` — range: other
  - RMSE computed exclusively along liquid-vapor bubble interfaces to penalize misalignment at sharp physical boundaries.

## Input / output format

**Input**: Optical flow: consecutive image frames tracking bubble positions. SciML: past solution fields (velocity and/or temperature) at k consecutive timesteps.

**Output**: Optical flow: 2D velocity vector field per pixel in Middlebury flow format. SciML: predicted temperature and/or velocity field at the next timestep.

## Scoring recipe

```python
def compute_epe(pred_flow, gt_flow):
    return np.mean(np.sqrt(np.sum((pred_flow - gt_flow)**2, axis=-1)))

def compute_rmse(pred, gt):
    return np.sqrt(np.mean((pred - gt)**2))

def compute_irmse(pred, gt, interface_mask):
    pred_masked = pred[interface_mask]
    gt_masked = gt[interface_mask]
    return np.sqrt(np.mean((pred_masked - gt_masked)**2))
```

## Common pitfalls

- High error rates at bubble boundaries due to sharp temperature/velocity gradients and non-rigid deformation.
- Auto-regressive rollout models suffer from error accumulation over time, degrading long-horizon predictions.
- Over-fitting to the boiling dataset during fine-tuning harms generalization to other optical flow benchmarks.

## Evidence (verbatim from paper)

> To assess the performance of the trained models, we measure the end-point error and Table 2 summarizes the results for one dataset. We draw inspiration from PDEBench and adopt a large set of metrics that include the Root Mean Squared Error (RMSE), Max Squared Error, Relative Error, Boundary RMSE (BRMSE), and low/mid/high Fourier errors. We incorporate an additional physics metric: the RMSE along bubble interfaces (IRMSE).

## Citation

```bibtex
@misc{hassan2023bubbleml,
  title={BubbleML: A Multi-Physics Dataset and Benchmarks for Machine Learning},
  author={Sheikh Md Shakeel Hassan et al. (2023)},
  year={2023},
  note={arXiv:2307.14623}
}
```

- arXiv: 2307.14623