Cloth Unfolding Grasp Eval

---
name: cloth-unfolding-grasp-eval
description: Evaluates a robot's ability to select effective grasp poses on hanging, folded cloth to maximize unfolding coverage. It probes material-aware perception, geometric reasoning, and policy-based grasp generation under complex folding configurations. Use when the user wants to benchmark on ICRA 2024 Cloth Competition dataset, or asks about evaluating this task. Reports relative coverage.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.16749
  bibtex_key: degusseme2025clothcompetition
  confidence: medium
---

# cloth-unfolding-grasp-eval

> A Dataset and Benchmark for Robotic Cloth Unfolding Grasp Selection: The ICRA 2024 Cloth Competition — Victor-Louis De Gusseme et al. (2025) (ICRA 2024 / arXiv:2508.16749, 2025)

## What this evaluates

Evaluates a robot's ability to select effective grasp poses on hanging, folded cloth to maximize unfolding coverage. It probes material-aware perception, geometric reasoning, and policy-based grasp generation under complex folding configurations.

## Datasets

- **ICRA 2024 Cloth Competition dataset** — total 679; splits: train (503), test (176); repo https://github.com/Victorlouisdg/cloth-competition

## Metrics

- `relative coverage` **(primary)** — range: [0, 1]
  - Ratio of the unfolded cloth's surface area to the maximum achievable surface area (measured from a reference fully unfolded observation). Success rate is also tracked as a binary metric based on coverage thresholds or manual annotation.

## Input / output format

**Input**: Stereo RGB images, depth and confidence maps, colored 3D point cloud, robot joint angles, gripper/base poses, camera intrinsics/extrinsics, and video. Each instance provides a start observation of the hanging cloth and the executed grasp pose.

**Output**: Grasp pose (position and orientation) for the robot gripper, typically specified as a 4x4 transformation matrix or coordinate frame relative to the world/cloth.

## Scoring recipe

```python
def score_grasp(result_obs, reference_obs):
    cloth_area = compute_surface_area(result_obs.point_cloud)
    max_area = compute_surface_area(reference_obs.point_cloud)
    coverage = cloth_area / max_area if max_area > 0 else 0.0
    success = coverage > 0.5  # threshold inferred from competition context
    return {'coverage': coverage, 'success': success}
```

## Common pitfalls

- Training videos are sometimes incomplete due to recording issues, so evaluation should rely on the 176 fully recorded competition trials.
- The dataset intentionally includes failed grasps and low-coverage attempts; users must filter or account for these rather than assuming all demonstrations are successful.
- Depth-based segmentation is required to remove robot arms and background for accurate surface area calculation, which may vary across different robotic setups.

## Evidence (verbatim from paper)

> The dataset includes a reference observation for each cloth item. This observation shows the cloth fully unfolded and held in the air, providing a baseline measurement of the maximum achievable surface area for use in evaluation metrics that measure relative coverage.

## Citation

```bibtex
@misc{degusseme2025clothcompetition,
  title={A Dataset and Benchmark for Robotic Cloth Unfolding Grasp Selection: The ICRA 2024 Cloth Competition},
  author={Victor-Louis De Gusseme et al. (2025)},
  year={2025},
  note={ICRA 2024 / arXiv:2508.16749}
}
```

- arXiv: 2508.16749