Omnispectra Eval

---
name: omnispectra-eval
description: Evaluates a foundation model's ability to process native-resolution astronomical spectra of variable lengths without resampling, and assesses its zero-shot, few-shot, and supervised performance on stellar property estimation and source classification tasks across diverse spectroscopic surveys. Use when the user wants to benchmark on OmniSpectra Multi-Survey Corpus, or asks about evaluating this task. Reports Mean-Squared Error.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.15351
  bibtex_key: islam2026omnispectra
  confidence: high
---

# omnispectra-eval

> OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra — Islam et al. (2026) (arXiv:2601.15351, 2026)

## What this evaluates

Evaluates a foundation model's ability to process native-resolution astronomical spectra of variable lengths without resampling, and assesses its zero-shot, few-shot, and supervised performance on stellar property estimation and source classification tasks across diverse spectroscopic surveys.

## Datasets

- **OmniSpectra Multi-Survey Corpus** — total 5543000; splits: train (5488000), val (55000)

## Metrics

- `Mean-Squared Error` **(primary)** — range: other
  - Standard mean squared error calculated as the average of the squared differences between predicted and true property values: MSE = (1/N) * Σ(y_true - y_pred)^2. Lower values indicate better performance. The paper reports a weighted average across multiple stellar targets.

## Input / output format

**Input**: Variable-length astronomical spectra provided as HDF5 files, tokenized via overlapping patches of 20 flux points (10-point overlap), with each instance independently normalized. Wavelengths are encoded sinusoidally.

**Output**: Continuous numerical values for stellar/galactic properties (e.g., temperature, metallicity) or class labels for source classification, depending on the downstream task.

## Scoring recipe

```python
def compute_mse(y_true, y_pred, weights=None):
    squared_diffs = (y_true - y_pred) ** 2
    if weights is not None:
        mse = np.average(squared_diffs, weights=weights)
    else:
        mse = np.mean(squared_diffs)
    return mse
```

## Common pitfalls

- Downstream evaluation splits are randomly sampled (20k examples, 50/50 train-test) per dataset rather than using standard held-out survey splits, which may not reflect real-world domain shift.
- The validation set (55k spectra) is a fixed 1% random subsample of the pretraining corpus, not a separate downstream validation set.
- Variable-length inputs require adaptive patching; models that force fixed-length resampling will lose native resolution fidelity.

## Evidence (verbatim from paper)

> Table 3: Mean-Squared Error (↓) of star property estimation on the SDSS dataset and its subsets. We report the weighted average across the targets. The best and second-best results are in bold and underline. OmniSpectra significantly outperforms other foundation models and even custom supervised models with little tuning.

## Citation

```bibtex
@misc{islam2026omnispectra,
  title={OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra},
  author={Islam et al. (2026)},
  year={2026},
  note={arXiv:2601.15351}
}
```

- arXiv: 2601.15351