Modified Median Absolute Deviation

---
name: modified_median_absolute_deviation
description: Evaluates the precision and accuracy of stellar flux recovery and positional measurements in simulated Roman Space Telescope images. It probes how well effective PSF models can recover input fluxes and coordinates across different spatial grids, filters, and magnitudes. Use when the user has predictions and gold and needs to compute modified median absolute deviation ($\hat{\sigma}$).
metadata:
  skill_kind: metric
  source_arxiv: 2506.04332
  bibtex_key: [REDACTED]
  confidence: high
---

# modified_median_absolute_deviation

> Initial Characterization of Stellar Photometry of Roman images from the OpenUniverse Simulations — Aldoroty et al. (2025) (arXiv:2506.04332, 2025)

## What this evaluates

Evaluates the precision and accuracy of stellar flux recovery and positional measurements in simulated Roman Space Telescope images. It probes how well effective PSF models can recover input fluxes and coordinates across different spatial grids, filters, and magnitudes.

## Datasets

- **OpenUniverse2024** — total ?; splits: test (-1); repo https://github.com/Roman-SupernovaPIT/pub-aldoroty-2025a

## Metrics

- `modified median absolute deviation ($\hat{\sigma}$)` **(primary)** — range: [0, 1]
  - 1.48 times the median of the absolute fractional flux difference between truth and fit fluxes. Assumes a Gaussian distribution for the fractional errors.
- `non-linearity slope ($s_{NL}$)` — range: other
  - Slope of the fractional flux recovery versus magnitude (per dex) in the non-saturated, sky noise-subdominant region ($19 < m_{truth} < 21.5$).

## Input / output format

**Input**: Simulated stellar images from OpenUniverse2024 with known truth coordinates and fluxes, filtered by magnitude ($19 < m_{truth} < 21.5$), bandpass, and spatial grid size ($1\times1$ to $8\times8$ SCAs).

**Output**: Fitted flux ($F_{fit}$), fitted pixel coordinates ($x_{fit}, y_{fit}$), and derived statistical metrics ($\hat{\sigma}$, $s_{NL}$, $s_{color}$, $\alpha$) computed over binned populations.

## Scoring recipe

```python
frac_flux_err = (F_truth - F_fit) / F_truth
sigma_hat = 1.48 * np.median(np.abs(frac_flux_err))
s_NL = np.polyfit(np.log10(m_truth), frac_flux_err, 1)[0]
s_color = np.polyfit(J129_fit - H158_fit, frac_flux_err, 1)[0]
A = [sum(quadrant[i]) for i in range(4)]
alpha = (1/4) * sum(abs(A[i] - A[j]) / (A[i] + A[j]) for i in range(4) for j in range(i+1, 4))
```

## Common pitfalls

- Pixel-phase bias in undersampled filters (R062, Z087) causes systematic coordinate offsets and flux scatter.
- PSF asymmetry and ellipticity are not fully captured by standard ePSF models, leading to filter-dependent recovery errors.
- Non-linearity measured here stems from photometric recovery rather than detector CRNL, requiring separate calibration.

## Evidence (verbatim from paper)

> We calculate a modified median absolute deviation  $(\hat{\sigma})$  of the fractional flux difference between truth and fit fluxes (Section 2.3) for these stars,  $(F_{\mathrm{truth}} - F_{\mathrm{fit}}) / F_{\mathrm{truth}}$ , defined as

$$
\hat {\sigma} = 1. 4 8 \times \mathrm {m e d i a n} \bigg (\Big | \frac {F _ {\mathrm {t r u t h}} - F _ {\mathrm {f i t}}}{F _ {\mathrm {t r u t h}}} \Big | \bigg), \qquad (1)

$$

where the factor of 1.48 comes from an assumption that  $(F_{\mathrm{truth}} - F_{\mathrm{fit}}) / F_{\mathrm{truth}}$  has a Gaussian distribution.

## Citation

```bibtex
@misc{aldoroty2025initial,
  title={Initial Characterization of Stellar Photometry of Roman images from the OpenUniverse Simulations},
  author={Aldoroty et al. (2025)},
  year={2025},
  note={arXiv:2506.04332}
}
```

- arXiv: 2506.04332