Photometric Redshift Estimation Eval

---
name: photometric-redshift-estimation-eval
description: Evaluates the accuracy of predicting galaxy photometric redshifts from optical (grizy) photometric data across multiple redshift ranges. It probes a model's ability to minimize systematic bias, reduce catastrophic outliers, and maintain low error rates under varying data distributions. Use when the user wants to benchmark on Hyper Suprime-Cam Photometric Redshift Data, or asks about evaluating this task. Reports MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.07292
  bibtex_key: biswas2026photometric
  confidence: high
---

# photometric-redshift-estimation-eval

> Photometric Redshift Estimation Using Scaled Ensemble Learning — Biswas et al. (2026) (arXiv:2601.07292, 2026)

## What this evaluates

Evaluates the accuracy of predicting galaxy photometric redshifts from optical (grizy) photometric data across multiple redshift ranges. It probes a model's ability to minimize systematic bias, reduce catastrophic outliers, and maintain low error rates under varying data distributions.

## Datasets

- **Hyper Suprime-Cam Photometric Redshift Data** — total ?; splits: test (-1)

## Metrics

- `MAE` **(primary)** — range: other
  - Mean Absolute Error: average of absolute differences between predicted and true redshift values.
- `rms` — range: other
  - Root Mean Square Error: square root of the average of squared differences between predicted and true redshift values.
- `bias` — range: other
  - Mean Bias: average difference between predicted and true redshift values (z_pred - z_true).
- `O_c` — range: percent
  - Catastrophic outlier rate: fraction of predictions falling outside asymmetric tolerance boundaries defined by y_pred = 1.15*y_test + 0.15 and y_pred = 0.85*y_test - 0.15.

## Input / output format

**Input**: Optical photometric fluxes in grizy bands for individual galaxies.

**Output**: Predicted photometric redshift (z_pred) as a continuous scalar value.

## Scoring recipe

```python
def score(z_true, z_pred):
    mae = np.mean(np.abs(z_pred - z_true))
    rmse = np.sqrt(np.mean((z_pred - z_true)**2))
    bias = np.mean(z_pred - z_true)
    upper = 1.15 * z_true + 0.15
    lower = 0.85 * z_true - 0.15
    oc = np.mean((z_pred > upper) | (z_pred < lower))
    return {'MAE': mae, 'rms': rmse, 'bias': bias, 'O_c': oc}
```

## Common pitfalls

- Tolerance boundaries for catastrophic outliers are asymmetric and redshift-dependent, not the standard symmetric fractional error used in many ML benchmarks.
- The dataset exhibits a strong redshift distribution bias toward lower z (z < 0.5), which can skew aggregate metrics if not stratified or weighted properly.
- Bagging the input data yields a 3-6% improvement across all metrics; evaluating on unbagged data will significantly underestimate model performance.

## Evidence (verbatim from paper)

> The performance metrics observed here are mean average error (MAE), rms, bias, and catastrophic outlier. The MAE or the loss is evaluated using Equation (17). The other performance metrics are evaluated using Equations (18)-(21.

## Citation

```bibtex
@misc{biswas2026photometric,
  title={Photometric Redshift Estimation Using Scaled Ensemble Learning},
  author={Biswas et al. (2026)},
  year={2026},
  note={arXiv:2601.07292}
}
```

- arXiv: 2601.07292