Blast Forecasting Eval

---
name: blast-forecasting-eval
description: Evaluates the zero-shot forecasting capability of universal time series models across multiple domains and prediction horizons. It probes how well pre-trained models generalize to unseen datasets and measures predictive accuracy using standard error metrics. Use when the user wants to benchmark on ETTh1, ETTh2, ETTm1, ETTm2, Weather, GlobalTemp, GIFT-Eval, or asks about evaluating this task. Reports MSE, MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.17871
  bibtex_key: shao2025blast
  confidence: high
---

# blast-forecasting-eval

> BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models — Zezhi Shao et al. (arXiv:2505.17871, 2025)

## What this evaluates

Evaluates the zero-shot forecasting capability of universal time series models across multiple domains and prediction horizons. It probes how well pre-trained models generalize to unseen datasets and measures predictive accuracy using standard error metrics.

## Datasets

- **ETTh1, ETTh2, ETTm1, ETTm2, Weather, GlobalTemp** — total ?; splits: test (-1)
- **GIFT-Eval** — total 43; splits: test (43)

## Metrics

- `MSE` **(primary)** — range: other
  - Normalized Mean Squared Error between predicted and actual values.
- `MAE` **(primary)** — range: other
  - Mean Absolute Error between predicted and actual values.
- `MASE` — range: other
  - Mean Absolute Scaled Error relative to a naive baseline forecast, following the GIFT-Eval protocol.

## Input / output format

**Input**: Historical time series sequences provided in a zero-shot setting, without task-specific fine-tuning.

**Output**: Predicted future values for specified forecasting horizons (96, 192, 336, or 720 time steps).

## Scoring recipe

```python
def compute_metrics(preds, targets, horizon):
    mse = np.mean((preds - targets) ** 2)
    mae = np.mean(np.abs(preds - targets))
    naive_pred = targets[:-horizon]
    mase = np.mean(np.abs(preds - targets)) / np.mean(np.abs(targets[horizon:] - naive_pred))
    return {'MSE': mse, 'MAE': mae, 'MASE': mase}
```

## Common pitfalls

- Data leakage: must filter out any time series segments already present in the pre-training corpus (Time-300B, LOTSA, BLAST) before evaluation.
- Horizon-specific reporting: metrics must be evaluated and reported separately for each prediction length (96, 192, 336, 720).
- Zero-shot constraint: models are evaluated without task-specific fine-tuning; results should not reflect adapter or prompt-tuning gains unless explicitly stated.

## Evidence (verbatim from paper)

> We report the normalized Mean Squared Error (MSE) and Mean Absolute Error (MAE). For the GIFT-Eval benchmark*(Aksu et al., [2024])*, we filtered out data already included in Time-300B (TimeMoE pre-training data), LOTSA (MOIRAI pre-training data), and BLAST, and strictly followed its evaluation pipeline. We report the Mean Absolute Scaled Error (MASE).

## Citation

```bibtex
@misc{shao2025blast,
  title={BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models},
  author={Zezhi Shao et al.},
  year={2025},
  note={arXiv:2505.17871}
}
```

- arXiv: 2505.17871