Panmatch Eval

---
name: panmatch-eval
description: Evaluates a unified vision model's ability to perform correspondence matching across stereo disparity estimation, optical flow, and feature matching in a zero-shot setting. It probes cross-domain generalization and robustness to challenging conditions like occlusion, lighting changes, and non-Lambertian surfaces. Use when the user wants to benchmark on Middlebury, ETH3D, KITTI, Infinigen, Spring, Sintel, Booster, or asks about evaluating this task. Reports PCA x.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.08400
  bibtex_key: [REDACTED]
  confidence: high
---

# panmatch-eval

> PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models — Zhang et al. (2025) (arXiv:2507.08400, 2025)

## What this evaluates

Evaluates a unified vision model's ability to perform correspondence matching across stereo disparity estimation, optical flow, and feature matching in a zero-shot setting. It probes cross-domain generalization and robustness to challenging conditions like occlusion, lighting changes, and non-Lambertian surfaces.

## Datasets

- **Middlebury** — total ?; splits: train-h (-1), half (-1), quarter (-1)
- **ETH3D** — total ?; splits: train (-1)
- **KITTI** — total ?; splits: 2012 (-1), 2015 (-1)
- **Infinigen** — total ?; splits: val (-1)
- **Spring** — total ?; splits: val (-1)
- **Sintel** — total ?; splits: clean (-1), final (-1)
- **Booster** — total ?; splits: test (-1)

## Metrics

- `PCA x` **(primary)** — range: percent
  - Percentage of Accuracy in threshold x pixel. Computes the fraction of valid pixels where the absolute displacement error is below a dataset-specific threshold (1px for ETH3D, 2px for Middlebury, 3px for KITTI, 1px for Infinigen/Spring).
- `EPE` — range: pixels
  - Endpoint Error. The mean L2 distance between predicted and ground-truth displacement vectors across all valid pixels.
- `Bad x` — range: percent
  - Percentage of pixels where the absolute error exceeds threshold x (e.g., Bad 1.0, Bad 2.0).
- `D1-all` — range: percent
  - Percentage of pixels with disparity error greater than 1 pixel or 5% of the ground truth disparity.

## Input / output format

**Input**: Paired two-view images (left/right for stereo, consecutive frames for flow, or arbitrary pairs for feature matching).

**Output**: Dense displacement field (optical flow), disparity map, or dense correspondence map.

## Scoring recipe

```python
def compute_pca(pred, gt, threshold):
    valid = gt > 0
    error = np.abs(pred[valid] - gt[valid])
    return np.mean(error < threshold) * 100

def compute_epe(pred, gt):
    valid = gt != 0
    error = np.sqrt(np.sum((pred[valid] - gt[valid])**2, axis=-1))
    return np.mean(error)
```

## Common pitfalls

- Thresholds for PCA vary by dataset (1px ETH3D, 2px Middlebury, 3px KITTI).
- Feature matching evaluation requires forward-backward circular consistency filtering before pose estimation.
- Zero-shot models are compared against RVC methods that are fine-tuned on benchmark training splits, making direct leaderboard ranking misleading for real-world robustness.

## Evidence (verbatim from paper)

> For stereo matching, we assess disparity accuracy on the Middlebury (train-h), ETH3D (train), and KITTI 2012 (train) datasets. We use percentage of accuracy in threshold x pixel (PCA x) as the evaluation metric, setting threshold 1px for ETH3D, 2px for Middlebury and 3px for KITTI, respectively.

## Citation

```bibtex
@misc{zhang2025panmatch,
  title={PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2507.08400}
}
```

- arXiv: 2507.08400