Crowd Pose Estimation Eval

---
name: crowd-pose-estimation-eval
description: Evaluates the ability of pose estimation models to accurately predict 2D keypoints for humans and animals in crowded, occluded, and multi-instance scenarios. It probes robustness to detection ambiguity, overlapping instances, and the transferability of conditional pose inputs from bottom-up detectors to top-down refiners. Use when the user wants to benchmark on CrowdPose, OCHuman, COCO, Multi-Animal (SchoolingFish, Marmosets, Tri-Mouse), or asks about evaluating this task. Reports AP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2306.07879
  bibtex_key: zhou2023rethinking
  confidence: high
---

# crowd-pose-estimation-eval

> Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity — Mu Zhou et al. (2023) (arXiv:2306.07879, 2023)

## What this evaluates

Evaluates the ability of pose estimation models to accurately predict 2D keypoints for humans and animals in crowded, occluded, and multi-instance scenarios. It probes robustness to detection ambiguity, overlapping instances, and the transferability of conditional pose inputs from bottom-up detectors to top-down refiners.

## Datasets

- **CrowdPose** — total 20000; splits: trainval (12000), test (8000)
- **OCHuman** — total 4731; splits: test (4731)
- **COCO** — total 82000; splits: train (57000), val (5000), test-dev (20000)
- **Multi-Animal (SchoolingFish, Marmosets, Tri-Mouse)** — total ?; splits: test (-1)

## Metrics

- `AP` **(primary)** — range: [0, 1]
  - Average Precision computed over IoU thresholds (typically 0.5:0.95) for matching predicted keypoints to ground truth within a normalized distance threshold. Variants include APeasy/APmed/APhard (crowd density/occlusion splits) and APM/APL (object size splits).
- `APeasy` — range: [0, 1]
  - AP computed only on easy instances (low crowd density/occlusion).
- `APmed` — range: [0, 1]
  - AP computed only on medium instances.
- `APhard` — range: [0, 1]
  - AP computed only on hard instances (high crowd density/occlusion).
- `APM` — range: [0, 1]
  - AP computed only on medium-sized objects.
- `APL` — range: [0, 1]
  - AP computed only on large-sized objects.

## Input / output format

**Input**: RGB images of crowded scenes, optionally paired with conditional bounding boxes or 2D pose predictions from a bottom-up model.

**Output**: Predicted 2D keypoints (e.g., 14 for humans) with instance IDs and confidence scores for each detected person or animal.

## Scoring recipe

```python
def compute_pose_ap(predictions, ground_truth, iou_thresh=0.5):
    matches = []
    for gt in ground_truth:
        best_score = 0
        for pred in predictions:
            if pred.instance_id == gt.instance_id:
                iou = calculate_keypoint_iou(pred, gt)
                if iou >= iou_thresh:
                    best_score = max(best_score, pred.confidence)
        matches.append(best_score)
    return average_precision(matches)
```

## Common pitfalls

- Using standard object detector bounding boxes instead of bottom-up pose predictions as conditional inputs significantly degrades performance in crowded scenes.
- Training on COCO and evaluating on OCHuman introduces a domain shift that requires careful validation, as models may overfit to COCO's less crowded distribution.
- Failing to account for generative vs. empirical sampling during training leads to poor generalization when swapping bottom-up model inputs at inference time.

## Evidence (verbatim from paper)

> We report standard metrics AP, APeasy, APmed and APhard as defined in[[26]]. We compared our method, that derives bounding boxes from a bottom-up model (see Methods), with baselines that used bounding boxes obtained by a Faster R-CNN detector[[37]].

## Citation

```bibtex
@misc{zhou2023rethinking,
  title={Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity},
  author={Mu Zhou et al. (2023)},
  year={2023},
  note={arXiv:2306.07879}
}
```

- arXiv: 2306.07879