Human Flow Eval

---
name: human-flow-eval
description: Evaluates the accuracy and inference speed of optical flow estimation networks specifically for human motion. It probes a model's ability to capture fine-grained, small-scale displacements typical of human limbs and body parts against complex or layered backgrounds. Use when the user wants to benchmark on Human Flow, or asks about evaluating this task. Reports AEPE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1806.05666
  bibtex_key: ranjan2018learninghumanopticalflow
  confidence: high
---

# human-flow-eval

> Learning Human Optical Flow — Ranjan et al. (2018) (arXiv:1806.05666, 2018)

## What this evaluates

Evaluates the accuracy and inference speed of optical flow estimation networks specifically for human motion. It probes a model's ability to capture fine-grained, small-scale displacements typical of human limbs and body parts against complex or layered backgrounds.

## Datasets

- **Human Flow** — total ?; splits: train (135153), test (10867)

## Metrics

- `AEPE` **(primary)** — range: pixels
  - Average End Point Error: the mean L2 Euclidean distance between predicted and ground truth optical flow vectors across all pixels in the test set.
- `Evaluation Time` — range: seconds
  - Wall-clock time in seconds required to process a single pair of frames on an NVIDIA Titan X GPU.

## Input / output format

**Input**: Pair of consecutive RGB frames (images) containing human subjects in motion.

**Output**: 2D optical flow field (H x W x 2 tensor) representing pixel-wise displacement vectors between the two frames.

## Scoring recipe

```python
def compute_aepe(pred_flow, gt_flow):
    # pred_flow and gt_flow are HxWx2 tensors
    diff = pred_flow - gt_flow
    l2_norms = torch.sqrt(torch.sum(diff**2, dim=-1) + 1e-6)
    return torch.mean(l2_norms).item()
```

## Common pitfalls

- Models trained on general datasets (e.g., FlowNetS) often fail on the small motions typical of human limbs, yielding results close to a zero-flow baseline.
- Layered optical flow methods (e.g., PCA-layers) heavily depend on accurate foreground-background segmentation, which is frequently imperfect in complex scenes.
- Traditional non-learning methods (e.g., LDOF, EpicFlow) tend to produce blurred flow fields that miss sharp boundaries at hands and legs, and are significantly slower than real-time inference.

## Evidence (verbatim from paper)

> We compare the average End Point Errors (EPEs) of competing methods in Table 1(b) along with the time for evaluation. Human motion is complex and general optical flow methods fail to capture it. Our trained network outperforms previous methods, and SPyNet [5] in particular, in terms of average EPE on the Human Flow Dataset.

## Citation

```bibtex
@misc{ranjan2018learninghumanopticalflow,
  title={Learning Human Optical Flow},
  author={Ranjan et al. (2018)},
  year={2018},
  note={arXiv:1806.05666}
}
```

- arXiv: 1806.05666