Wildbe Eval

---
name: wildbe-eval
description: Evaluates object detection algorithms on drone-captured images of wild berries in cluttered, dynamic forest environments. It probes localization and classification capabilities under severe lighting variations, occlusion, and cross-domain transfer settings (different areas, cameras, and datasets). Use when the user wants to benchmark on WildBe, or asks about evaluating this task. Reports Average Precision (AP).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2405.07550
  bibtex_key: [REDACTED]
  confidence: high
---

# wildbe-eval

> Wild Berry image dataset collected in Finnish forests and peatlands using drones — Riz et al. (2024) (arXiv:2405.07550, 2024)

## What this evaluates

Evaluates object detection algorithms on drone-captured images of wild berries in cluttered, dynamic forest environments. It probes localization and classification capabilities under severe lighting variations, occlusion, and cross-domain transfer settings (different areas, cameras, and datasets).

## Datasets

- **WildBe** — total 3516; splits: train (3164), val (176), test (176)

## Metrics

- `Average Precision (AP)` **(primary)** — range: percent
  - COCO evaluation metric computing precision-recall curve over IoU thresholds (0.50:0.05:0.95) and averaging AP across thresholds. Also reported for small (≤32×32 px) and medium (32×32 to 96×96 px) objects.
- `Instance-weighted Average Precision (WAvg)` — range: percent
  - Average AP weighted by the number of ground-truth instances per class, used to mitigate performance masking due to class imbalance in multi-class settings.

## Input / output format

**Input**: RGB images captured by drones, annotated with bounding boxes and class labels (bilberry, cloudberry, crowberry, lingonberry, or generic 'berry').

**Output**: Bounding box coordinates and predicted class labels for each detected object in the image.

## Scoring recipe

```python
coco_gt = load_annotations(gold_boxes, gold_classes)
coco_dt = load_predictions(pred_boxes, pred_classes, scores)
coco_eval = COCOeval(coco_gt, coco_dt, 'bbox')
coco_eval.evaluate()
coco_eval.accumulate()
ap = coco_eval.summaries[0]  # AP@[.50:.95]
ap_s = coco_eval.summaries[1]  # AP small
ap_m = coco_eval.summaries[2]  # AP medium
class_aps = [coco_eval.summaries[0] for cid in classes]
wavg = sum(ap * count for ap, count in zip(class_aps, class_counts)) / sum(class_counts)
```

## Common pitfalls

- No large detections (≥96×96 px) exist in WildBe, so AP_large is always 0 or undefined.
- WAvg is explicitly recommended over simple Avg AP when class distributions are highly imbalanced (e.g., bilberry vs. lingonberry) to prevent class bias.
- Transfer learning folds are defined by geographic area or camera sensor, not random image splits; models must be trained on 3/4 of folds and tested on the held-out fold.

## Evidence (verbatim from paper)

> We use COCO evaluation and report results in terms of Average Precision (AP). For the single-class experiments, we also report the AP for small ($	extrm{AP}_{	extrm{S}}$) and medium ($	extrm{AP}_{	extrm{M}}$) detections. COCO defines detections as “small” if they occupy up to $32	imes 32$ pixels, “medium” if they occupy between $32	imes 32$ and $96	imes 96$ pixels, “large” otherwise. No large detections are present in WildBe. For the multi-class experiments, we report the per-class AP, the average AP (Avg), and the instance-weighted average AP (WAvg).

## Citation

```bibtex
@misc{riz2024wildberry,
  title={Wild Berry image dataset collected in Finnish forests and peatlands using drones},
  author={Riz et al. (2024)},
  year={2024},
  note={arXiv:2405.07550}
}
```

- arXiv: 2405.07550