Relative Mean Square Error

---
name: relative-mean-square-error
description: Evaluates the prediction accuracy of data-driven neural network closures for kinetic theory of active fluids against reference kinetic simulations and traditional analytical closures. It probes the model's ability to capture rotational symmetries, generalize across parameter regimes, and resolve topological defects in fluid dynamics. Use when the user has predictions and gold and needs to compute RMSE.
metadata:
  skill_kind: metric
  source_arxiv: 2308.06675
  bibtex_key: maddu2023learning
  confidence: high
---

# relative-mean-square-error

> Learning fast, accurate, and stable closures of a kinetic theory of an active fluid — Maddu et al. (2023) (arXiv:2308.06675, 2023)

## What this evaluates

Evaluates the prediction accuracy of data-driven neural network closures for kinetic theory of active fluids against reference kinetic simulations and traditional analytical closures. It probes the model's ability to capture rotational symmetries, generalize across parameter regimes, and resolve topological defects in fluid dynamics.

## Datasets

- **Kinetic simulation data for active fluid closures** — total ?; splits: train (-1), test (-1)

## Metrics

- `RMSE` **(primary)** — range: [0, 1]
  - Relative Mean Square Error computed as the squared L2 norm of the difference between reference and predicted closure terms, divided by the squared L2 norm of the reference term. Formula: RMSE(ζ) = ||S:T(ζ) - S̃:T(ζ,θ)||₂² / ||S:T(ζ)||₂².

## Input / output format

**Input**: Invariant quantities I_s(D,E) or independent tensor components {d11,d12,d22,e11,e12,e22}, along with alignment strength ζ and other parameters (α, β, dT, dR).

**Output**: Predicted closure term S̃:T (tensor contraction value).

## Scoring recipe

```python
def compute_rmse(reference, prediction):
    num = np.linalg.norm(reference - prediction)**2
    den = np.linalg.norm(reference)**2
    return num / den
```

## Common pitfalls

- FNO architectures show significantly higher error with component-wise learning but improve by two orders of magnitude when using invariant representations.
- Extrapolation accuracy drops sharply when testing on ζ values outside the training range due to overfitting and changes in fluid flow characteristics.
- Traditional closures (Linear, Quadratic) and FNO struggle to resolve prediction errors near topological defects compared to MLP/CNN with invariant features.

## Evidence (verbatim from paper)

> We use relative mean-square error (RMSE) as the performance metric: RMSE(ζ) = ||S:T(ζ) - S̃:T(ζ,θ)||₂² / ||S:T(ζ)||₂². From Fig. 3, we report that, irrespective of the network architecture, learning based on invariant representation leads to better performance in terms of prediction accuracy compared to component-wise learning.

## Citation

```bibtex
@misc{maddu2023learning,
  title={Learning fast, accurate, and stable closures of a kinetic theory of an active fluid},
  author={Maddu et al. (2023)},
  year={2023},
  note={arXiv:2308.06675}
}
```

- arXiv: 2308.06675