Mobility Timeseries Eval

---
name: mobility-timeseries-eval
description: Evaluates the accuracy of time series forecasting models on urban mobility data across different prediction horizons. It probes how well traditional, deep learning, and foundation models capture short-term, medium-term, and long-term temporal dependencies in bike-sharing flows. Use when the user wants to benchmark on BikeNYC, BikeVIE, or asks about evaluating this task. Reports RMSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.03725
  bibtex_key: graser2025timeseries
  confidence: high
---

# mobility-timeseries-eval

> Timeseries Foundation Models for Mobility: A Benchmark Comparison with Traditional and Deep Learning Models — Graser (2025) (arXiv:2504.03725, 2025)

## What this evaluates

Evaluates the accuracy of time series forecasting models on urban mobility data across different prediction horizons. It probes how well traditional, deep learning, and foundation models capture short-term, medium-term, and long-term temporal dependencies in bike-sharing flows.

## Datasets

- **BikeNYC** — total ?; splits: test (-1)
- **BikeVIE** — total ?; splits: test (-1)

## Metrics

- `RMSE` **(primary)** — range: other
  - Root Mean Square Error, calculated as the square root of the mean of squared differences between predicted and actual values across the forecast horizon.

## Input / output format

**Input**: Hourly time series of bike-sharing counts (flows or station availability) for each grid cell or station.

**Output**: Forecasted bike-sharing counts for 1-hour, 12-hour, and 24-hour prediction horizons.

## Scoring recipe

```python
def compute_rmse(predictions, actuals):
    n = len(predictions)
    squared_errors = [(p - a) ** 2 for p, a in zip(predictions, actuals)]
    return (sum(squared_errors) / n) ** 0.5
```

## Common pitfalls

- Uses a rolling window backtesting approach on only the last 10 days, which may not represent full seasonal cycles.
- BikeVIE dataset has a hard cutoff at mid-August 2019 to avoid seasonal gaps, limiting long-term evaluation.
- Forecast horizons (1h, 12h, 24h) are evaluated separately, so models must be compared at each horizon independently.

## Evidence (verbatim from paper)

> Performance is evaluated using Root Mean Square Error (RMSE) for 1-hour, 12-hour, and 24-hour forecasts, with the last ten days used for backtesting via a rolling window approach.

## Citation

```bibtex
@misc{graser2025timeseries,
  title={Timeseries Foundation Models for Mobility: A Benchmark Comparison with Traditional and Deep Learning Models},
  author={Graser (2025)},
  year={2025},
  note={arXiv:2504.03725}
}
```

- arXiv: 2504.03725