Exoplanet Imaging Challenge Ii Eval

---
name: exoplanet-imaging-challenge-ii-eval
description: This benchmark evaluates the ability of high-contrast imaging post-processing pipelines to accurately estimate the astrometric position of injected exoplanet signals in multispectral astronomical data. It probes how well algorithms handle varying signal-to-noise ratios, complex residual backgrounds (e.g., diffraction patterns, coronagraphic inner working angles), and different observing conditions. Use when the user wants to benchmark on Exoplanet Imaging Data Challenge Phase II, or asks about evaluating this task. Reports D_astro^GT.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2410.17636
  bibtex_key: cantalloube2024exoplanetimaging
  confidence: high
---

# exoplanet-imaging-challenge-ii-eval

> Exoplanet Imaging Data Challenge, phase II: Comparison of algorithms in terms of characterization capabilities — Cantalloube et al. (2024) (arXiv:2410.17636, 2024)

## What this evaluates

This benchmark evaluates the ability of high-contrast imaging post-processing pipelines to accurately estimate the astrometric position of injected exoplanet signals in multispectral astronomical data. It probes how well algorithms handle varying signal-to-noise ratios, complex residual backgrounds (e.g., diffraction patterns, coronagraphic inner working angles), and different observing conditions.

## Datasets

- **Exoplanet Imaging Data Challenge Phase II** — total ?; splits: test (-1)

## Metrics

- `D_astro^GT` **(primary)** — range: [0, ∞)
  - Euclidean distance (L2-norm) between the estimated Cartesian coordinates of the injected planetary signal and its ground-truth position.

## Input / output format

**Input**: High-contrast multispectral images from GPI or SPHERE-IFS instruments containing injected planetary signals at known ground-truth locations, along with metadata on observing conditions.

**Output**: Estimated position (Cartesian coordinates or separation/position angle) for each injected planetary signal, optionally accompanied by 1σ uncertainties and the posterior distribution used for estimation.

## Scoring recipe

```python
def compute_astrometry_error(predictions, ground_truth):
    # predictions and ground_truth are 2D arrays of (x, y) coordinates
    diff = predictions - ground_truth
    l2_distances = np.sqrt(np.sum(diff**2, axis=1))
    return l2_distances
```

## Common pitfalls

- Proximity to bright diffraction features (e.g., coronagraph IWA, spider diffraction, low wind effect) significantly distorts flux distribution and biases position estimation, often overriding the impact of global observing conditions.
- Low signal-to-noise ratio (SNR) increases estimation uncertainty and error, but spatial location within the field of view frequently dominates accuracy over overall data quality.
- Classical Gaussian fitting methods (e.g., ANDROMEDA) fail to account for complex residual backgrounds, leading to large astrometric biases that exceed one resolution element.

## Evidence (verbatim from paper)

> D_astro^GT is the metric chosen for the astrometry (L2-norm distance between the estimation and the ground-truth).

## Citation

```bibtex
@misc{cantalloube2024exoplanetimaging,
  title={Exoplanet Imaging Data Challenge, phase II: Comparison of algorithms in terms of characterization capabilities},
  author={Cantalloube et al. (2024)},
  year={2024},
  note={arXiv:2410.17636}
}
```

- arXiv: 2410.17636