Bayesian Optical Flow Eval

---
name: bayesian-optical-flow-eval
description: Evaluates a Bayesian statistical inversion method for estimating optical flow fields and quantifying their uncertainty from image pairs, compared against deterministic baselines. Use when the user wants to benchmark on Synthetic benchmark flow fields, Middlebury dataset, or asks about evaluating this task. Reports reconstruction accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1611.01230
  bibtex_key: sun2016bayesian
  confidence: high
---

# bayesian-optical-flow-eval

> Bayesian Optical Flow with Uncertainty Quantification — Sun et al. (2016) (arXiv:1611.01230, 2016)

## What this evaluates

Evaluates a Bayesian statistical inversion method for estimating optical flow fields and quantifying their uncertainty from image pairs, compared against deterministic baselines.

## Datasets

- **Synthetic benchmark flow fields** — total 5; splits: test (5)
- **Middlebury dataset** — total 6; splits: test (6)

## Metrics

- `reconstruction accuracy` **(primary)** — range: other
  - Quantified via scatter plot correlation between the reconstructed second image ($\hat{G}$) derived from the estimated mean flow, and both the noisy reference image ($G$) and the noiseless ground-truth image ($\bar{G}$). Evaluated qualitatively by visual comparison of mean flow fields against true flows.

## Input / output format

**Input**: Grayscale image pair (F, G) at 30x30 (synthetic) or 60x60 (real) resolution, with pixel intensities normalized to [0,1].

**Output**: Posterior distribution of flow fields p(U,V) via MCMC sampling; mean flow field estimate and pixel-wise uncertainty confidence regions.

## Scoring recipe

```python
# 1. Sample posterior p(U,V) using MCMC-Gibbs
samples = mcmc_gibbs_sample(image_pair, priors, hyperpriors)
# 2. Compute mean flow field
mean_flow = np.mean(samples, axis=0)
# 3. Reconstruct second image from mean flow
G_hat = reconstruct_second_image(F, mean_flow)
# 4. Compare G_hat to G (noisy) and G_bar (noiseless) via scatter plot correlation
corr_noisy = pearsonr(G_hat.flatten(), G.flatten())
corr_noiseless = pearsonr(G_hat.flatten(), G_bar.flatten())
```

## Common pitfalls

- MCMC sampling may fail to converge to a stationary distribution; requires restarting from different initial seeds.
- Uncertainty quantification relies on fitting a 2D normal distribution to MCMC samples after discarding initial transients.
- Evaluation is primarily qualitative/visual rather than relying on a single standardized scalar metric.

## Evidence (verbatim from paper)

> To test the usefulness of the reconstructed flow field, we use it together with the first image F to obtain an estimated second image \hat{G} from equation (38), setting the noise term to be zero. The estimated second image \hat{G} is shown for each example in the third column of Fig.[7] through Fig.[12]. Interestingly, the estimated \hat{G} seems to not only resemble the given noisy image G, but even more similar to the noiseless second image \bar{G}. This observation is confirmed quantitatively, as shown in the last column of Fig.[7] through Fig.[12], where the values of \hat{G}_{ij} are plotted against both those of G_{ij} and of \bar{G}_{ij}.

## Citation

```bibtex
@misc{sun2016bayesian,
  title={Bayesian Optical Flow with Uncertainty Quantification},
  author={Sun et al. (2016)},
  year={2016},
  note={arXiv:1611.01230}
}
```

- arXiv: 1611.01230