Park Cleaning Benchmark

---
name: park-cleaning-benchmark
description: Evaluates autonomous cleaning robots' ability to navigate public park pathways, perceive and collect diverse litter types, avoid obstacles, and operate within strict physical and safety constraints. Use when the user wants to benchmark on Park Cleaning Benchmark, or asks about evaluating this task. Reports collected_items_weight_or_count.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.16393
  bibtex_key: memmesheimer2024cleaningrobots
  confidence: medium
---

# park-cleaning-benchmark

> Cleaning Robots in Public Spaces: A Survey and Proposal for Benchmarking Based on Stakeholders Interviews — Memmesheimer et al. (2024) (arXiv:2407.16393, 2024)

## What this evaluates

Evaluates autonomous cleaning robots' ability to navigate public park pathways, perceive and collect diverse litter types, avoid obstacles, and operate within strict physical and safety constraints.

## Datasets

- **Park Cleaning Benchmark** — total ?; splits: test (-1)

## Metrics

- `collected_items_weight_or_count` **(primary)** — range: kg or count or seconds
  - Total weight (kg) or count of successfully collected items from the pathway and wayside. Alternatively, total time (seconds) to complete the cleaning task.

## Input / output format

**Input**: Real-world park environment containing a pathway, adjacent wayside (up to 1m from path), various litter types (glass, plastic cups, paper, cigarette butts) with varying size/material/moisture, and static/dynamic obstacles (curbs, cycle barriers, benches, parked cars).

**Output**: Autonomous navigation and manipulation actions to collect litter while avoiding obstacles and adhering to platform constraints (≤150kg, ≤1.2×0.75×1m, emergency button, obstacle avoidance).

## Scoring recipe

```python
def score(collected_items, task_time):
    # Primary metric: weight or count of collected items
    score = sum(item.weight for item in collected_items)
    # Bonus for hard-to-reach litter
    for item in collected_items:
        if item.location == 'hard_to_reach':
            score += bonus_points
    return score
```

## Common pitfalls

- Proposed as a conceptual scenario without released code or fixed dataset splits
- Scoring metric is ambiguous: paper allows either collected item weight/count OR completion time
- Technical acceptance (size/weight limits, emergency button, obstacle avoidance) must pass before benchmark scoring begins

## Evidence (verbatim from paper)

> The benchmark scoring considers the amount of collected items (in weight or amount) or the time required to achieve the cleaning task. ... The technical requirements are defined as follows: Weight ≤ 150kg, Size ≤ 1.2m×0.75m×1m (L×W×H), Well visible and reachable emergency button, Obstacle avoidance (avoiding people, animals, pathway users and other obstacles)

## Citation

```bibtex
@misc{memmesheimer2024cleaningrobots,
  title={Cleaning Robots in Public Spaces: A Survey and Proposal for Benchmarking Based on Stakeholders Interviews},
  author={Memmesheimer et al. (2024)},
  year={2024},
  note={arXiv:2407.16393}
}
```

- arXiv: 2407.16393