Tempusbench Univariate Eval

---
name: tempusbench-univariate-eval
description: Evaluates time-series foundation models, statistical methods, and machine learning algorithms on univariate forecasting tasks. It probes their ability to handle diverse statistical properties like stationarity, seasonality, sparsity, and noise across real-world and synthetic datasets. Use when the user wants to benchmark on TempusBench Univariate Benchmark, or asks about evaluating this task. Reports MASE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.11529
  bibtex_key: goktas2026tempusbench
  confidence: medium
---

# tempusbench-univariate-eval

> TempusBench: An Evaluation Framework for Time-Series Forecasting — Goktas et al. (2026) (arXiv:2604.11529, 2026)

## What this evaluates

Evaluates time-series foundation models, statistical methods, and machine learning algorithms on univariate forecasting tasks. It probes their ability to handle diverse statistical properties like stationarity, seasonality, sparsity, and noise across real-world and synthetic datasets.

## Datasets

- **TempusBench Univariate Benchmark** — total ?; splits: test (-1); repo https://github.com/Smlcrm/TempusBench

## Metrics

- `MASE` **(primary)** — range: [0, ∞)
  - Mean Absolute Scaled Error; a value below 1.0 indicates the model outperforms a naive baseline forecast.

## Input / output format

**Input**: Univariate time series sequences (single variable) with historical observations.

**Output**: Point forecasts for the target horizon.

## Scoring recipe

```python
def compute_mase(y_true, y_pred, y_train):
    naive_errors = np.abs(y_train[1:] - y_train[:-1])
    scale = np.mean(naive_errors)
    forecast_errors = np.abs(y_true - y_pred)
    return np.mean(forecast_errors) / scale
```

## Common pitfalls

- Evaluating models without standardized hyperparameter tuning across statistical, ML, and DL types creates unfair performance comparisons.
- Ignoring core statistical properties like stationarity and seasonality when selecting or interpreting benchmark tasks leads to misleading generalization claims.
- Using a naive baseline that does not match the series' seasonal period distorts MASE interpretation.

## Evidence (verbatim from paper)

> The MASE (Mean Absolute Scaled Error) results reveal distinct performance patterns for single time series forecasting. TimesFM emerges as the most consistent top performer, achieving MASE < 1.0 (better than naive baseline) on 8 out of 26 tasks, including standout performances on Electricity (0.79971) and Soil Nature (0.27718).

## Citation

```bibtex
@misc{goktas2026tempusbench,
  title={TempusBench: An Evaluation Framework for Time-Series Forecasting},
  author={Goktas et al. (2026)},
  year={2026},
  note={arXiv:2604.11529}
}
```

- arXiv: 2604.11529