Sku 110k Detection Eval

---
name: sku-110k-detection-eval
description: Evaluates object detection and counting capabilities in densely packed scenes, specifically testing a model's ability to localize and count tightly overlapping items without false positives from standard non-maximum suppression. Use when the user wants to benchmark on SKU-110K, CARPK, PUCPR+, or asks about evaluating this task. Reports AP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1904.00853
  bibtex_key: goldman2019precise
  confidence: high
---

# sku-110k-detection-eval

> Precise Detection in Densely Packed Scenes — Goldman et al. (2019) (arXiv:1904.00853, 2019)

## What this evaluates

Evaluates object detection and counting capabilities in densely packed scenes, specifically testing a model's ability to localize and count tightly overlapping items without false positives from standard non-maximum suppression.

## Datasets

- **SKU-110K** — total ?; splits: train (-1), test (-1)
- **CARPK** — total ?; splits: test (-1)
- **PUCPR+** — total ?; splits: test (-1)

## Metrics

- `AP` **(primary)** — range: [0, 1]
  - Average Precision computed over predicted bounding boxes and ground truth annotations, typically averaged across IoU thresholds.
- `AP.75` — range: [0, 1]
  - Average Precision computed strictly at an Intersection over Union (IoU) threshold of 0.75.
- `AR300` — range: [0, 1]
  - Average Recall calculated with a maximum of 300 detections allowed per image.
- `PR=.5` — range: [0, 1]
  - Precision measured at a recall threshold of 0.5 (or IoU=0.5 depending on convention).
- `MAE` — range: other
  - Mean Absolute Error between the predicted object count and the ground truth count per image.
- `RMSE` — range: other
  - Root Mean Square Error between the predicted object count and the ground truth count per image.

## Input / output format

**Input**: RGB images of densely packed retail shelves or high-angle parking lot scenes.

**Output**: A list of bounding boxes with associated confidence or Soft-IoU scores, and optionally a single integer representing the total object count per image.

## Scoring recipe

```python
def compute_metrics(pred_boxes, pred_scores, gt_boxes, gt_counts):
    # Detection metrics via standard IoU matching and PR curve integration
    ap, ap75, ar300, pr5 = compute_detection_metrics(pred_boxes, pred_scores, gt_boxes)
    # Counting metrics
    mae = np.mean(np.abs(np.array(pred_counts) - np.array(gt_counts)))
    rmse = np.sqrt(np.mean((np.array(pred_counts) - np.array(gt_counts))**2))
    return {'AP': ap, 'AP.75': ap75, 'AR300': ar300, 'PR=.5': pr5, 'MAE': mae, 'RMSE': rmse}
```

## Common pitfalls

- Standard NMS incorrectly suppresses valid detections for tightly packed/overlapping objects.
- Objectness confidence scores are less reliable than Soft-IoU scores for filtering false positives.
- Counting accuracy degrades significantly more than detection accuracy when using greedy NMS instead of overlap-aware merging.

## Evidence (verbatim from paper)

> Accuracy is reported using MAE and RMSE, as in our SKU-110K (Sec.[4]).

## Citation

```bibtex
@misc{goldman2019precise,
  title={Precise Detection in Densely Packed Scenes},
  author={Goldman et al. (2019)},
  year={2019},
  note={arXiv:1904.00853}
}
```

- arXiv: 1904.00853