Lstnet Time Series Forecasting Eval

---
name: lstnet-time-series-forecasting-eval
description: Evaluates multivariate time series forecasting models on their ability to capture short-term local dependencies and long-term periodic patterns across diverse real-world datasets with varying temporal scales and frequencies. Use when the user wants to benchmark on Traffic, Solar-Energy, Electricity, Exchange-Rate, or asks about evaluating this task. Reports RSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1703.07015
  bibtex_key: lai2017lstnet
  confidence: high
---

# lstnet-time-series-forecasting-eval

> Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks — Lai et al. (2017) (arXiv:1703.07015, 2017)

## What this evaluates

Evaluates multivariate time series forecasting models on their ability to capture short-term local dependencies and long-term periodic patterns across diverse real-world datasets with varying temporal scales and frequencies.

## Datasets

- **Traffic** — total 17544; splits: train (-1), val (-1), test (-1)
- **Solar-Energy** — total 52560; splits: train (-1), val (-1), test (-1)
- **Electricity** — total 26304; splits: train (-1), val (-1), test (-1)
- **Exchange-Rate** — total 7588; splits: train (-1), val (-1), test (-1)

## Metrics

- `RSE` **(primary)** — range: [0, 1] or higher
  - Root Relative Squared Error: RSE = sqrt(Σ(Y_it - Ŷ_it)^2) / sqrt(Σ(Y_it - mean(Y))^2). It is a scale-insensitive version of RMSE normalized by a baseline mean prediction. Lower values indicate better performance.
- `CORR` — range: [-1, 1]
  - Empirical Correlation Coefficient: CORR = (1/n) Σ_i [ Σ_t((Y_it - mean(Y_i))(Ŷ_it - mean(Ŷ_i))) / sqrt(Σ_t(Y_it - mean(Y_i))^2 * Σ_t(Ŷ_it - mean(Ŷ_i))^2) ]. It measures the average Pearson correlation across all D variables. Higher values indicate better performance.

## Input / output format

**Input**: Multivariate time series sequences of length T with D variables, sampled at fixed intervals (hourly, 10-min, or daily). Models receive historical windows and predict future values over horizons of 3, 6, 12, or 24 time steps depending on the dataset.

**Output**: Predicted values for D variables over the specified forecasting horizon.

## Scoring recipe

```python
def compute_rse(y_true, y_pred, y_mean):
    num = np.sqrt(np.sum((y_true - y_pred)**2))
    den = np.sqrt(np.sum((y_true - y_mean)**2))
    return num / den

def compute_corr(y_true, y_pred):
    n = y_true.shape[0]
    total = 0.0
    for i in range(n):
        yt, yp = y_true[i], y_pred[i]
        myt, myp = np.mean(yt), np.mean(yp)
        cov = np.sum((yt - myt) * (yp - myp))
        denom = np.sqrt(np.sum((yt - myt)**2) * np.sum((yp - myp)**2))
        total += cov / denom
    return total / n
```

## Common pitfalls

- Forecasting horizons are dataset-specific (hours for Traffic/Electricity, minutes for Solar-Energy, days for Exchange-Rate), so direct cross-dataset metric comparison is invalid without horizon normalization.
- Single-output baselines (AR, LRidge, LSVR, GP) are trained independently per variable, whereas multivariate models share parameters; this architectural difference affects fairness in comparison.
- RSE is scale-insensitive and lower is better, whereas CORR is higher is better; confusing the optimization direction leads to incorrect model selection.

## Evidence (verbatim from paper)

> We used three conventional evaluation metrics defined as: Root Relative Squared Error (RSE): ... Empirical Correlation Coefficient (CORR) ... where Y, Ŷ ∈ R^{n×T} are ground true signals and system prediction signals, respectively. ... All datasets have been split into training set (60%), validation set (20%) and test set (20%) in chronological order.

## Citation

```bibtex
@misc{lai2017lstnet,
  title={Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks},
  author={Lai et al. (2017)},
  year={2017},
  note={arXiv:1703.07015}
}
```

- arXiv: 1703.07015