Deco 50 Eval

---
name: deco-50-eval
description: Evaluates a robot policy's ability to perform bimanual dexterous manipulation tasks under varying levels of tactile dependency. It probes visual-propriocceptive coordination, dynamic object interaction, and contact-rich force control. Use when the user wants to benchmark on DECO-50, or asks about evaluating this task. Reports Success Rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.05513
  bibtex_key: li2026deco
  confidence: high
---

# deco-50-eval

> DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter — Li et al. (2026) (arXiv:2602.05513, 2026)

## What this evaluates

Evaluates a robot policy's ability to perform bimanual dexterous manipulation tasks under varying levels of tactile dependency. It probes visual-propriocceptive coordination, dynamic object interaction, and contact-rich force control.

## Datasets

- **DECO-50** — total ?; splits: train (-1), test (-1)

## Metrics

- `Success Rate` **(primary)** — range: percent
  - Calculated as the number of successful task completions divided by the total number of attempts. Two aggregate metrics are reported: Success Rate 1 (average across all four tasks) and Success Rate 2 (average across the two contact-rich tasks: Waste Disposal and Assembly). For Waste Disposal, success requires completing all three sequential stages.

## Input / output format

**Input**: Synchronized multimodal observations: active binocular camera images, tactile sensor signals, and robot proprioceptive state.

**Output**: Continuous bimanual action commands (e.g., joint positions or end-effector poses) for the dual-arm robot.

## Scoring recipe

```python
def compute_success_rate(successes, attempts):
    return successes / attempts

def compute_aggregate_sr(task_success_rates):
    return sum(task_success_rates) / len(task_success_rates)

# Success Rate 1: average over all 4 tasks
# Success Rate 2: average over Waste Disposal and Assembly
```

## Common pitfalls

- Assuming tactile sensing is universally required; the paper demonstrates it only significantly boosts performance on contact-rich tasks (Waste Disposal, Assembly).
- Misinterpreting aggregate metrics: Success Rate 1 averages all tasks, while Success Rate 2 specifically averages only the contact-rich tasks.
- Overlooking stage-wise dependencies in Waste Disposal, where overall success requires completing all three sequential stages (open lid, pick/throw, close lid).

## Evidence (verbatim from paper)

> Success Rate 1: Average success rate on all tasks. Success Rate 2: Average success rate on contact-rich tasks, which are Waste Disposal and Assembly.

## Citation

```bibtex
@misc{li2026deco,
  title={DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter},
  author={Li et al. (2026)},
  year={2026},
  note={arXiv:2602.05513}
}
```

- arXiv: 2602.05513