Shirt Pose Tracking Eval

---
name: shirt-pose-tracking-eval
description: Evaluates the accuracy and robustness of a neural network-integrated Unscented Kalman Filter for monocular pose tracking of tumbling noncooperative spacecraft. It probes the system's ability to maintain steady-state position and orientation accuracy under domain gaps between synthetic training data and real hardware-in-the-loop test images. Use when the user wants to benchmark on SHIRT, or asks about evaluating this task. Reports e_pose.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2206.03796
  bibtex_key: park2022adaptive
  confidence: high
---

# shirt-pose-tracking-eval

> Adaptive Neural Network-based Unscented Kalman Filter for Robust Pose Tracking of Noncooperative Spacecraft — Park et al. (2022) (arXiv:2206.03796, 2022)

## What this evaluates

Evaluates the accuracy and robustness of a neural network-integrated Unscented Kalman Filter for monocular pose tracking of tumbling noncooperative spacecraft. It probes the system's ability to maintain steady-state position and orientation accuracy under domain gaps between synthetic training data and real hardware-in-the-loop test images.

## Datasets

- **SHIRT** — total ?; splits: test (-1)

## Metrics

- `e_pose` **(primary)** — range: other
  - Normalized pose error computed as e_t / ||t|| + e_q, where e_t is translation error, ||t|| is the ground-truth translation norm, and e_q is rotation error in radians. Mean pose error E_pose is the average over N images.

## Input / output format

**Input**: Sequential monocular images of a target spacecraft from a servicer's camera, initial pose predictions from SPNv2, and absolute orbital/attitude states of the servicer.

**Output**: Time-series estimates of relative translation, relative velocity, and relative attitude/angular velocity states.

## Scoring recipe

```python
def compute_mean_pose_error(preds, gts):
    total = 0.0
    for p, g in zip(preds, gts):
        e_t = np.linalg.norm(p.trans - g.trans)
        e_q = rotation_error_rad(p.rot, g.rot)
        norm_t = np.linalg.norm(g.trans)
        total += (e_t / norm_t) + e_q
    return total / len(preds)
```

## Common pitfalls

- Rotation error e_q must be in radians, not degrees or quaternion distance.
- Translation error must be normalized by ground-truth distance ||t||; failing to do so biases evaluation toward close-range encounters.
- Initial process noise covariance Q_o requires manual tuning before ASNC activates; poor initialization causes filter divergence regardless of adaptive updates.

## Evidence (verbatim from paper)

> For individual images, filter performance is evaluated using the translation error $(e_{\mathrm{t}})$ and rotation error $(e_{\mathrm{q}})$ defined in Eq. 1. Additionally, the pose error from SPEC2021 [16] is used as a singular metric wherever applicable and is given as $$e _ {\text {p o s e}} = e _ {\mathrm {t}} / \| \boldsymbol {t} \| + e _ {\mathrm {q}}$$ where $t$ is the ground-truth translation vector of the sample, and $e_{\mathrm{q}}$ is in radians. For batches of images, the mean translation, rotation, and pose errors are reported, respectively defined as $$E _ {\mathrm {t}} = \sum _ {i = 1 } ^ {N} e _ {\mathrm {t}} ^ {(i)}$$ $$E _ {\mathrm {q}} = \sum _ {i = 1 } ^ {N} e _ {\mathrm {q}} ^ {(i)}$$ $$E _ {\text {p o s e}} = \sum _ {i = 1 } ^ {N} e _ {\text {p o s e}} ^ {(i)}$$

## Citation

```bibtex
@misc{park2022adaptive,
  title={Adaptive Neural Network-based Unscented Kalman Filter for Robust Pose Tracking of Noncooperative Spacecraft},
  author={Park et al. (2022)},
  year={2022},
  note={arXiv:2206.03796}
}
```

- arXiv: 2206.03796