Window Wise Complexity

---
name: window-wise-complexity
description: Quantifies the intrinsic predictability of time series data by measuring window-wise pattern complexity in the frequency domain. It establishes a data-driven performance lower bound for forecasting models and identifies whether standard benchmarks have reached saturation. Use when the user has predictions and gold and needs to compute window-wise complexity.
metadata:
  skill_kind: metric
  source_arxiv: 2510.02729
  bibtex_key: wang2025accuracylaw
  confidence: high
---

# window-wise-complexity

> Accuracy Law for the Future of Deep Time Series Forecasting — Wang et al. (2025) (arXiv:2510.02729, 2025)

## What this evaluates

Quantifies the intrinsic predictability of time series data by measuring window-wise pattern complexity in the frequency domain. It establishes a data-driven performance lower bound for forecasting models and identifies whether standard benchmarks have reached saturation.

## Datasets

- **ETT** — total ?; splits: test (-1)
- **Electricity** — total ?; splits: test (-1)
- **Weather** — total ?; splits: test (-1)
- **Exchange-Rate** — total ?; splits: test (-1)
- **Traffic** — total ?; splits: test (-1)

## Metrics

- `window-wise complexity` **(primary)** — range: other
  - Measures pattern complexity in the frequency domain over sliding windows of length (P+F), combining both past observation and forecasting horizon windows. Computed as the average amplitude spectrum difference across windows.
- `Pearson coefficient` — range: [-1, 1]
  - Standard linear correlation coefficient between the computed window-wise complexity and the best achieved forecasting error/performance on each series.

## Input / output format

**Input**: Univariate time series segments from benchmarks, partitioned into a past observation window (P) and a forecasting horizon (F).

**Output**: Scalar complexity score per series; best forecasting error achieved by deep models compared against the estimated lower bound.

## Scoring recipe

```python
def compute_window_wise_complexity(series, P, F):
    windows = [series[i:i+P+F] for i in range(0, len(series)-P-F+1)]
    amplitudes = [abs(fft(w)) for w in windows]
    return mean([norm(a) for a in amplitudes])

def check_saturation(series, model_error, complexity):
    lower_bound = estimate_lower_bound(complexity)
    return model_error < lower_bound
```

## Common pitfalls

- Relying on series-wise metrics (ADF, ForeCA, ACF half-life) instead of window-wise complexity yields weak correlation with actual forecasting performance.
- Assuming benchmarks are still challenging when performance gains are marginal; the metric reveals many are already saturated.
- Evaluating large models on in-domain data fails to expose pretraining-complexity imbalances and generalization limits.

## Evidence (verbatim from paper)

> Despite widespread adoption, we find that these series-wise predictability metrics do not produce a significant correlation with final forecasting performance in terms of the Pearson coefficient... the frequency domain measure significantly outperforms the time-domain DTW... nearly all points in ETT, Electricity, Weather, and Exchange-Rate benchmarks fall below the estimated performance lower bound, indicating that these datasets have likely reached their predictability limits.

## Citation

```bibtex
@misc{wang2025accuracylaw,
  title={Accuracy Law for the Future of Deep Time Series Forecasting},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2510.02729}
}
```

- arXiv: 2510.02729