Tamen Contact Rich Eval

---
name: tamen-contact-rich-eval
description: Probes a robot policy's ability to execute contact-rich bimanual manipulation tasks using visuo-tactile feedback. It evaluates robustness to visual disturbances, generalization to unseen object appearances, and the effectiveness of tactile pretraining and recovery data in imitation learning. Use when the user wants to benchmark on TAMEn Contact-Rich Manipulation Tasks, or asks about evaluating this task. Reports success rate (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.07335
  bibtex_key: wu2026tamen
  confidence: high
---

# tamen-contact-rich-eval

> TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks — Longyan Wu et al. (2026) (arXiv:2604.07335, 2026)

## What this evaluates

Probes a robot policy's ability to execute contact-rich bimanual manipulation tasks using visuo-tactile feedback. It evaluates robustness to visual disturbances, generalization to unseen object appearances, and the effectiveness of tactile pretraining and recovery data in imitation learning.

## Datasets

- **TAMEn Contact-Rich Manipulation Tasks** — total 80; splits: test (80)

## Metrics

- `success rate (%)` **(primary)** — range: percent
  - Calculated as the number of successful task executions divided by the total number of trials (20 per task), multiplied by 100. Success is defined by task-specific completion criteria (e.g., herbs successfully poured, cable fully seated, clip detached, stain removed).

## Input / output format

**Input**: RGB images from wrist-mounted fisheye cameras and tactile sensor readings from four fingertip visuo-tactile sensors.

**Output**: Continuous 16-DoF action vector controlling two 7-DoF arms and two DH grippers.

## Scoring recipe

```python
def compute_success_rate(predictions, gold, task_name):
    total_trials = 20
    successful = 0
    for trial in range(total_trials):
        if task_completed(task_name, predictions[trial]):
            successful += 1
    return (successful / total_trials) * 100
```

## Common pitfalls

- Confusing the large-scale pretraining dataset (FreeTacMan, 3M+ pairs) with the evaluation protocol, which uses only 20 trials per task.
- Overlooking the two distinct visual disturbance settings (full episode vs. post-grasp), which yield different success rates and should be reported separately.
- Assuming vision-only baselines are evaluated identically; tactile baselines require synchronized visuo-tactile input, and success often hinges on contact-rich stages where vision fails.

## Evidence (verbatim from paper)

> Each task in[Fig.˜6] is trained and evaluated over 20 trials. As shown in[Tab.˜III], incorporating tactile sensing improves the average success rate from 34% to 55%.

## Citation

```bibtex
@misc{wu2026tamen,
  title={TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks},
  author={Longyan Wu et al. (2026)},
  year={2026},
  note={arXiv:2604.07335}
}
```

- arXiv: 2604.07335