Urban Tree Detection Eval

---
name: urban-tree-detection-eval
description: Evaluates the capability of segmentation models to detect and delineate individual tree crowns and canopy coverage from high-resolution aerial imagery across diverse urban and tropical environments. Use when the user wants to benchmark on Zurich Municipal Tree Inventory & Swisstopo Imagery, WeRobotics Open AI Challenge (Tonga), or asks about evaluating this task. Reports Recall.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.11743
  bibtex_key: veitchmichaelis2024oamtcd
  confidence: high
---

# urban-tree-detection-eval

> OAM-TCD: A globally diverse dataset of high-resolution tree cover maps — Veitch-Michaelis et al. (2024) (arXiv:2407.11743, 2024)

## What this evaluates

Evaluates the capability of segmentation models to detect and delineate individual tree crowns and canopy coverage from high-resolution aerial imagery across diverse urban and tropical environments.

## Datasets

- **Zurich Municipal Tree Inventory & Swisstopo Imagery** — total 71000; splits: test (71000)
- **WeRobotics Open AI Challenge (Tonga)** — total 13402; splits: test (13402)

## Metrics

- `IoU` — range: [0, 1]
  - Intersection over Union: the area of overlap between predicted and ground truth canopy masks divided by the area of their union.
- `Accuracy` — range: [0, 1]
  - Proportion of correctly classified pixels (tree vs. non-tree) in the semantic segmentation mask.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall for semantic canopy segmentation.
- `Recall` **(primary)** — range: [0, 1]
  - Proportion of ground truth tree keypoints matched to either predicted tree instances or predicted canopy regions. TP / (TP + FN).

## Input / output format

**Input**: Tiled high-resolution aerial orthomosaics (10 cm/px resolution) processed in overlapping patches.

**Output**: Prediction maps for semantic canopy coverage and instance-level tree keypoints/polygons, with overlapping instances merged using heuristics.

## Scoring recipe

```python
# Semantic (Canopy)
iou = np.sum(pred & gt) / np.sum(pred | gt)
accuracy = np.mean(pred == gt)
f1 = 2 * (precision * recall) / (precision + recall)

# Instance/Keypoint
matched = 0
for gt_kp in gt_keypoints:
    if any(pred_inst.contains(gt_kp) or pred_canopy.contains(gt_kp)):
        matched += 1
recall = matched / len(gt_keypoints)
```

## Common pitfalls

- Models detect non-municipal trees not present in the ground truth inventory, making false positive rate calculation impossible.
- Tiled inference requires custom heuristics to merge overlapping predictions across tile boundaries.
- Clustered trees are often predicted as single groups, artificially lowering instance-level recall.

## Evidence (verbatim from paper)

> The Federal Institute of Topography in Switzerland (Swisstopo) produces 3-year aerial surveys of the entire country, at 10 cm resolution. By combining 2022 imagery with a up-to-date municipal tree inventory from the city of Zurich, we can assess model performance against an accurate ground truth. We report a recall of 0.36 for keypoints that were matched to tree labels and 0.73 when also including canopy regions. This dataset is a tree detection benchmark. The input is a 325 ha orthomosaic captured over the Kingdom of Tonga. Tree centers for four species were annotated by humans (13402 total keypoints); not all trees in the image are labelled and our model predicts around twice this many instances over the orthomosaic. We report a recall of 0.64 for keypoints that were matched to tree labels and 0.94 when also including canopy regions.

## Citation

```bibtex
@misc{veitchmichaelis2024oamtcd,
  title={OAM-TCD: A globally diverse dataset of high-resolution tree cover maps},
  author={Veitch-Michaelis et al. (2024)},
  year={2024},
  note={arXiv:2407.11743}
}
```

- arXiv: 2407.11743