Success Rate 3d Policy

---
name: success-rate-3d-policy
description: Evaluates the robustness and generalization of 3D policy learning models for robotic manipulation across varying environmental conditions, temporal horizons, and real-world interference. It probes spatial understanding, fine-grained pose control, and resilience to domain randomization and lighting changes. Use when the user wants to benchmark on RoboTwin 2.0, ManiSkill2, Real-World Manipulation, or asks about evaluating this task. Reports Success Rate (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.15281
  bibtex_key: hong2026r3d
  confidence: high
---

# success-rate-3d-policy

> R3D: Revisiting 3D Policy Learning — Hong et al. (2026) (arXiv:2604.15281, 2026)

## What this evaluates

Evaluates the robustness and generalization of 3D policy learning models for robotic manipulation across varying environmental conditions, temporal horizons, and real-world interference. It probes spatial understanding, fine-grained pose control, and resilience to domain randomization and lighting changes.

## Datasets

- **RoboTwin 2.0** — total ?; splits: Easy (-1), Hard (-1)
- **ManiSkill2** — total ?; splits: PegInsertionSide (-1), PickCube (-1), StackCube (-1)
- **Real-World Manipulation** — total ?; splits: Place Kettle (-1), Open Drawer (-1), Fold Towel (-1)

## Metrics

- `Success Rate (%)` **(primary)** — range: percent
  - Calculated as the percentage of successful trials out of the total number of trials per task. The paper specifies evaluating each data point on 50 trials for both simulation and real-world settings.

## Input / output format

**Input**: Cropped 3D point clouds (1024 points for simulation, 8192 for real-world) representing the environment, robot state, and goal positions. Real-world inputs may use single or dual camera views.

**Output**: Continuous action commands for robotic manipulation, decoded via a diffusion-based action decoder.

## Scoring recipe

```python
def compute_success_rate(trial_outcomes, total_trials=50):
    successes = sum(1 for outcome in trial_outcomes if outcome)
    return (successes / total_trials) * 100
```

## Common pitfalls

- Point cloud downsampling resolution differs between simulation (1024 points) and real-world (8192 points), which can artificially inflate or deflate performance if not standardized during evaluation.
- The PegInsertion task is evaluated across three sequential subtasks (Grasp, Align, Insert) rather than a single end-to-end metric, requiring separate tracking of each stage's success rate.
- Real-world evaluation includes a dynamic 'disco light' interference condition that specifically tests color robustness, which many baselines fail due to lighting-dependent point cloud coloring.

## Evidence (verbatim from paper)

> We report the success rate as the evaluation metric for these two benchmarks. Each data point is evaluated on 50 trials.

## Citation

```bibtex
@misc{hong2026r3d,
  title={R3D: Revisiting 3D Policy Learning},
  author={Hong et al. (2026)},
  year={2026},
  note={arXiv:2604.15281}
}
```

- arXiv: 2604.15281