Hierarchical Time Series Forecasting Eval

---
name: hierarchical-time-series-forecasting-eval
description: Evaluates the ability of spatiotemporal graph neural networks to perform multistep-ahead forecasting on correlated time series while simultaneously learning hierarchical cluster structures end-to-end. It probes the model's capacity to leverage relational inductive biases and self-supervised aggregation for improved prediction accuracy. Use when the user wants to benchmark on METR-LA, PEMS-BAY, AQI, CER-E, or asks about evaluating this task. Reports MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2305.19183
  bibtex_key: cini2023graph
  confidence: high
---

# hierarchical-time-series-forecasting-eval

> Graph-based Time Series Clustering for End-to-End Hierarchical Forecasting — Cini et al. (2023) (arXiv:2305.19183, 2023)

## What this evaluates

Evaluates the ability of spatiotemporal graph neural networks to perform multistep-ahead forecasting on correlated time series while simultaneously learning hierarchical cluster structures end-to-end. It probes the model's capacity to leverage relational inductive biases and self-supervised aggregation for improved prediction accuracy.

## Datasets

- **METR-LA** — total ?; splits: train/val/test (-1)
- **PEMS-BAY** — total ?; splits: train/val/test (-1)
- **AQI** — total ?; splits: train/val/test (-1)
- **CER-E** — total ?; splits: train/val/test (-1)

## Metrics

- `MAE` **(primary)** — range: other
  - Mean Absolute Error: average of absolute differences between predicted and true values over the forecasting horizon and all nodes.
- `MRE` — range: other
  - Mean Relative Error: average of relative differences between predicted and true values. Often computed as MAE divided by the mean of true values or element-wise relative error averaged.

## Input / output format

**Input**: Multivariate time series sequences of fixed length (input window) with associated graph adjacency matrices representing spatial correlations.

**Output**: Predicted multivariate time series values for a fixed forecasting horizon length.

## Scoring recipe

```python
def compute_mae(y_true, y_pred):
    return np.mean(np.abs(y_true - y_pred))

def compute_mre(y_true, y_pred):
    return np.mean(np.abs(y_true - y_pred) / np.abs(y_true))
```

## Common pitfalls

- Splits and graph topology extraction protocols are inherited from prior works ([24], [63], [22]) and not detailed in this paper.
- MRE definition is not explicitly formulaic in the text; readers should verify if it uses element-wise relative error or global mean normalization.
- Cluster quality is evaluated qualitatively via visualization (Fig. 3) rather than quantitative clustering metrics.

## Evidence (verbatim from paper)

> The performance metrics employed are the mean absolute error(MAE) and the mean relative error(MRE). The considered benchmarks comprise of 4 datasets in total and include 2 datasets from the traffic forecasting literature (METR-LA and PEMS-BAY), one dataset of air quality measurements (AQI) and a collection of energy consumption profiles (CER-E).

## Citation

```bibtex
@misc{cini2023graph,
  title={Graph-based Time Series Clustering for End-to-End Hierarchical Forecasting},
  author={Cini et al. (2023)},
  year={2023},
  note={arXiv:2305.19183}
}
```

- arXiv: 2305.19183