Evaluates time-series foundation models, statistical methods, and machine learning algorithms on univariate forecasting tasks. It probes their ability to handle diverse statistical properties like stationarity, seasonality, sparsity, and noise across real-world and synthetic datasets. Use when the user wants to benchmark on TempusBench Univariate Benchmark, or asks about evaluating this task. Reports MASE.
Installs into .claude/skills of the current project.
Are you the author of Tempusbench Univariate Eval?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/qhjqhj00-tempusbench-univariate-eval)
---
name: tempusbench-univariate-eval
description: Evaluates time-series foundation models, statistical methods, and machine learning algorithms on univariate forecasting tasks. It probes their ability to handle diverse statistical properties like stationarity, seasonality, sparsity, and noise across real-world and synthetic datasets. Use when the user wants to benchmark on TempusBench Univariate Benchmark, or asks about evaluating this task. Reports MASE.
metadata:
skill_kind: dataset_eval
source_arxiv: 2604.11529
bibtex_key: [REDACTED]
confidence: medium
---
# tempusbench-univariate-eval
> TempusBench: An Evaluation Framework for Time-Series Forecasting — Goktas et al. (2026) (arXiv:2604.11529, 2026)
## What this evaluates
Evaluates time-series foundation models, statistical methods, and machine learning algorithms on univariate forecasting tasks. It probes their ability to handle diverse statistical properties like stationarity, seasonality, sparsity, and noise across real-world and synthetic datasets.
## Datasets
- **TempusBench Univariate Benchmark** — total ?; splits: test (-1); repo https://github.com/Smlcrm/TempusBench
## Metrics
- `MASE` **(primary)** — range: [0, ∞)
- Mean Absolute Scaled Error; a value below 1.0 indicates the model outperforms a naive baseline forecast.
## Input / output format
**Input**: Univariate time series sequences (single variable) with historical observations.
**Output**: Point forecasts for the target horizon.
## Scoring recipe
```python
def compute_mase(y_true, y_pred, y_train):
naive_errors = np.abs(y_train[1:] - y_train[:-1])
scale = np.mean(naive_errors)
forecast_errors = np.abs(y_true - y_pred)
return np.mean(forecast_errors) / scale
```
## Common pitfalls
- Evaluating models without standardized hyperparameter tuning across statistical, ML, and DL types creates unfair performance comparisons.
- Ignoring core statistical properties like stationarity and seasonality when selecting or interpreting benchmark tasks leads to misleading generalization claims.
- Using a naive baseline that does not match the series' seasonal period distorts MASE interpretation.
## Evidence (verbatim from paper)
> The MASE (Mean Absolute Scaled Error) results reveal distinct performance patterns for single time series forecasting. TimesFM emerges as the most consistent top performer, achieving MASE < 1.0 (better than naive baseline) on 8 out of 26 tasks, including standout performances on Electricity (0.79971) and Soil Nature (0.27718).
## Citation
```bibtex
@misc{goktas2026tempusbench,
title={TempusBench: An Evaluation Framework for Time-Series Forecasting},
author={Goktas et al. (2026)},
year={2026},
note={arXiv:2604.11529}
}
```
- arXiv: 2604.11529
