Pap 12k Eval

---
name: pap-12k-eval
description: Evaluates a model's ability to predict affordance regions in 360° panoramic imagery. It probes spatial reasoning, handling of extreme scale variations, and robustness to geometric distortions inherent in equirectangular projection formats. Use when the user wants to benchmark on PAP-12K, or asks about evaluating this task. Reports gIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.15558
  bibtex_key: zhang2026panoramic
  confidence: high
---

# pap-12k-eval

> Panoramic Affordance Prediction — Zhang et al. (2026) (arXiv:2603.15558, 2026)

## What this evaluates

Evaluates a model's ability to predict affordance regions in 360° panoramic imagery. It probes spatial reasoning, handling of extreme scale variations, and robustness to geometric distortions inherent in equirectangular projection formats.

## Datasets

- **PAP-12K** — total 12000; splits: test (-1); repo https://github.com/EnVision-Research/PAP

## Metrics

- `gIoU` **(primary)** — range: percent
  - Average Intersection-over-Union across all test samples, measuring overall segmentation quality.
- `cIoU` — range: percent
  - Cumulative intersection over cumulative union across the entire dataset, providing a dataset-level quality measure less sensitive to individual object sizes.
- `P@50` — range: percent
  - Percentage of predictions with an IoU score exceeding 0.5, evaluating the model's ability to generate high-quality predictions.
- `P@50:95` — range: percent
  - Average precision calculated across a range of IoU thresholds from 0.5 to 0.95 with 0.05 increments, providing a stricter and more fine-grained assessment of segmentation accuracy.

## Input / output format

**Input**: Ultra-high-resolution (12K) 360° panoramic images in equirectangular projection format, accompanied by a textual affordance query/prompt.

**Output**: Binary affordance mask or segmentation map indicating the predicted region for the queried affordance.

## Scoring recipe

```python
def compute_metrics(pred_masks, gt_masks):
    ious = [iou(p, g) for p, g in zip(pred_masks, gt_masks)]
    gIoU = sum(ious) / len(ious) * 100
    cIoU = (sum(p.sum() for p in pred_masks) & sum(g.sum() for g in gt_masks)) / \
           (sum(p.sum() for p in pred_masks) | sum(g.sum() for g in gt_masks)) * 100
    P50 = sum(1 for i in ious if i > 0.5) / len(ious) * 100
    thresholds = [0.5 + 0.05 * i for i in range(10)]
    P50_95 = mean([sum(1 for i in ious if i >= t) / len(ious) for t in thresholds]) * 100
    return gIoU, cIoU, P50, P50_95
```

## Common pitfalls

- Standard 2D perspective models suffer severe performance degradation due to geometric distortion in equirectangular panoramic images.
- Objects split across the left/right image boundaries are frequently missed or incorrectly segmented by baseline methods.
- Extreme scale variations (objects <0.1% or >30% of image area) require multi-scale routing to avoid localization failure.

## Evidence (verbatim from paper)

> Following the standard evaluation protocol in affordance prediction (Wang et al., 2025a; Zhang et al., 2025c), we adopt four complementary metrics to comprehensively assess prediction quality, including: 1) $gIoU$ (Generalized IoU): The average Intersection-over-Union across all test samples, measuring the overall segmentation quality of the predicted affordance regions. 2) $cIoU$ (Cumulative IoU): The cumulative intersection over cumulative union across the entire dataset, providing a dataset-level quality measure that is less sensitive to the size of individual objects. 3) $P@50$ (Precision at IoU=0.5): The percentage of predictions with an IoU score exceeding 0.5, evaluating the model's ability to generate high-quality predictions. 4) $P@50:95$ : The average precision calculated across a range of IoU thresholds from 0.5 to 0.95 with 0.05 increments, providing a stricter and more fine-grained assessment of segmentation accuracy.

## Citation

```bibtex
@misc{zhang2026panoramic,
  title={Panoramic Affordance Prediction},
  author={Zhang et al. (2026)},
  year={2026},
  note={arXiv:2603.15558}
}
```

- arXiv: 2603.15558