Sim1 Tshirt Fold Eval

---
name: sim1-tshirt-fold-eval
description: Evaluates a robot policy's ability to perform structured deformable manipulation (t-shirt folding) in real-world settings after being trained exclusively on simulation data. It probes sim-to-real transfer, out-of-domain robustness to environmental shifts, and data scaling efficiency. Use when the user wants to benchmark on SIM1 T-shirt Folding, or asks about evaluating this task. Reports success.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.08544
  bibtex_key: zhou2026sim1
  confidence: high
---

# sim1-tshirt-fold-eval

> SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds — Zhou et al. (2026) (arXiv:2604.08544, 2026)

## What this evaluates

Evaluates a robot policy's ability to perform structured deformable manipulation (t-shirt folding) in real-world settings after being trained exclusively on simulation data. It probes sim-to-real transfer, out-of-domain robustness to environmental shifts, and data scaling efficiency.

## Datasets

- **SIM1 T-shirt Folding** — total ?; splits: test (30); repo https://github.com/InternRobotics/SIM1

## Metrics

- `success` **(primary)** — range: percent
  - Percentage of trials where the garment reaches the target folded configuration without dropping or unfolding. Calculated as (successful_trials / total_trials) * 100.

## Input / output format

**Input**: RGB camera views of the workspace, robot kinematic state, and garment configuration.

**Output**: Robot joint commands or end-effector trajectories for dual-arm manipulation.

## Scoring recipe

```python
def compute_success(predictions, gold):
    successful = 0
    for trial in predictions:
        if trial.reaches_target_folded_config() and not trial.drops_or_unfolds():
            successful += 1
    return (successful / len(predictions)) * 100
```

## Common pitfalls

- Confusing the 1:15 synthetic-to-real data equivalence ratio with absolute success rates.
- Failing to account for the π_0.5 vs π_0 pretraining difference, which causes π_0 to fail completely in low-data regimes.
- Not distinguishing between in-domain evaluation (identical setup) and out-of-domain generalization (spatial, texture, lighting, viewpoint shifts).

## Evidence (verbatim from paper)

> For the representative π_0.5 setting, real data reach average success 97%, while policies trained on sim-teleoperated data achieve 87%, a marginal gap of 10%. This suggests that physics-aligned simulation provides supervision of considerable fidelity to match real-world training with controlled data volumes.

## Citation

```bibtex
@misc{zhou2026sim1,
  title={SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds},
  author={Zhou et al. (2026)},
  year={2026},
  note={arXiv:2604.08544}
}
```

- arXiv: 2604.08544