Pkugoodsad Eval

---
name: pkugoodsad-eval
description: Evaluates unsupervised visual anomaly detection and segmentation models on real-world supermarket goods. It probes robustness to object misalignment, intra-class appearance variation, and the ability to detect subtle or small anomalies without labeled anomalous training data. Use when the user wants to benchmark on PKU-GoodsAD, or asks about evaluating this task. Reports AUROC, AUPR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2307.04956
  bibtex_key: zhang2023pkugoodsad
  confidence: high
---

# pkugoodsad-eval

> PKU-GoodsAD: A Supermarket Goods Dataset for Unsupervised Anomaly Detection and Segmentation — Jian Zhang et al. (2023) (arXiv:2307.04956, 2023)

## What this evaluates

Evaluates unsupervised visual anomaly detection and segmentation models on real-world supermarket goods. It probes robustness to object misalignment, intra-class appearance variation, and the ability to detect subtle or small anomalies without labeled anomalous training data.

## Datasets

- **PKU-GoodsAD** — total 6124; splits: train (-1), test (-1); repo https://github.com/jianzhang96/GoodsAD

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic curve. Measures the trade-off between true positive rate and false positive rate across classification thresholds.
- `AUPR` **(primary)** — range: [0, 1]
  - Area Under the Precision-Recall curve. More sensitive to class imbalance than AUROC, commonly used for anomaly detection where anomalies are rare.
- `PRO` — range: [0, 1]
  - Per-Region Overlap. Measures the overlap between predicted and ground truth anomaly regions, normalized by region size, to balance performance across different anomaly sizes.

## Input / output format

**Input**: RGB images of supermarket goods, resized to 224×224 pixels during training and testing. Ground truth pixel-level anomaly masks are provided for evaluation.

**Output**: Per-image anomaly score (for classification) and/or per-pixel anomaly score map (for segmentation/localization).

## Scoring recipe

```python
def compute_metrics(pred_scores, gt_labels, pred_masks=None, gt_masks=None):
    auroc = roc_auc_score(gt_labels, pred_scores)
    aupr = average_precision_score(gt_labels, pred_scores)
    pro = None
    if pred_masks is not None and gt_masks is not None:
        overlaps = []
        for p_mask, g_mask in zip(pred_masks, gt_masks):
            inter = np.sum(np.logical_and(p_mask, g_mask))
            union = np.sum(np.logical_or(p_mask, g_mask))
            if union > 0: overlaps.append(inter / union)
        pro = np.mean(overlaps) if overlaps else 0.0
    return {'AUROC': auroc, 'AUPR': aupr, 'PRO': pro}
```

## Common pitfalls

- Object location misalignment across images breaks methods that assume fixed object positions.
- High intra-class appearance variation makes it difficult for reconstruction/distillation models to learn normal distributions.
- Pseudo-anomaly generation techniques (e.g., CutPaste, DRAEM) produce artifacts that differ significantly from natural retail anomalies, leading to poor generalization.

## Evidence (verbatim from paper)

> The standard classification metrics AUROC and AUPR are used for image-level anomaly classification and pixel-level anomaly segmentation. AUPR is more sensitive to the datasets of unbalanced categories. PRO [[50]] is also adopted to balance anomalous areas of different sizes.

## Citation

```bibtex
@misc{zhang2023pkugoodsad,
  title={PKU-GoodsAD: A Supermarket Goods Dataset for Unsupervised Anomaly Detection and Segmentation},
  author={Jian Zhang et al. (2023)},
  year={2023},
  note={arXiv:2307.04956}
}
```

- arXiv: 2307.04956