Dial Turning Rl Eval

---
name: dial-turning-rl-eval
description: Evaluates a robot's ability to perform contact-based manipulation and learn a continuous control policy via reinforcement learning to match a target angle on a potentiometer. Use when the user wants to benchmark on DeltaZ Dial Turning Task, or asks about evaluating this task. Reports reward.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2207.00721
  bibtex_key: patil2022deltaz
  confidence: high
---

# dial-turning-rl-eval

> DeltaZ: An Accessible Compliant Delta Robot Manipulator for Research and Education — Patil et al. (2022) (arXiv:2207.00721, 2022)

## What this evaluates

Evaluates a robot's ability to perform contact-based manipulation and learn a continuous control policy via reinforcement learning to match a target angle on a potentiometer.

## Datasets

- **DeltaZ Dial Turning Task** — total ?; splits: test (-1); repo https://github.com/ZoomLabCMU/DeltaZ

## Metrics

- `reward` **(primary)** — range: other
  - A trial yields a reward of 100 if the final angle is within 15° of the desired angle, minus a quadratic penalty: R = 100[|φ-φd|<15] - 10^-5(φ-φd)^2. The policy terminates when all 10 trials in a batch achieve success.

## Input / output format

**Input**: Potentiometer resistance/angle reading and robot end-effector pose.

**Output**: Four continuous skill parameters (ρ1, θ1, ρ2, θ2) normalized to [-1, 1].

## Scoring recipe

```python
def compute_reward(final_angle, desired_angle):
    success = abs(final_angle - desired_angle) < 15
    reward = 100.0 if success else 0.0
    reward -= 1e-5 * (final_angle - desired_angle)**2
    return reward, success
```

## Common pitfalls

- The reward function includes a quadratic penalty that can make total reward negative even on successful trials, so reporting raw reward values without clarifying the success threshold is misleading.
- The task uses an automatic resetting mechanism and a fixed z-height, so results do not generalize to unstructured contact or variable approach heights.
- Policy convergence is evaluated on physical hardware with significant noise; simulation-to-real gaps are not addressed in this benchmark.

## Evidence (verbatim from paper)

> For each epsiode, the robot receives a reward of $R\=100$ if the final angle $\phi$ is within $15^{\circ}$ of the desired angle $\phi_{d}$, indicating a successful task completion, as well as a quadratic cost based on the difference between the final angle and the desired angle $R\=100[\|\phi-\phi_{d}\|<15]-10^{-5}(\phi-\phi_{d})^{2}$.

## Citation

```bibtex
@misc{patil2022deltaz,
  title={DeltaZ: An Accessible Compliant Delta Robot Manipulator for Research and Education},
  author={Patil et al. (2022)},
  year={2022},
  note={arXiv:2207.00721}
}
```

- arXiv: 2207.00721