Traffic Flow Forecasting Eval

---
name: traffic-flow-forecasting-eval
description: Evaluates the ability of spatiotemporal GNN models to forecast future traffic flow (speed or occupancy) based on historical sensor data and road network topology. Use when the user wants to benchmark on METR-LA, PEMS-BAY, PeMS04, or asks about evaluating this task. Reports MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2306.00012
  bibtex_key: li2023gnnspatiotemporal
  confidence: high
---

# traffic-flow-forecasting-eval

> Graph Neural Network for spatiotemporal data: methods and applications — Yun Li et al. (2023) (arXiv:2306.00012, 2023)

## What this evaluates

Evaluates the ability of spatiotemporal GNN models to forecast future traffic flow (speed or occupancy) based on historical sensor data and road network topology.

## Datasets

- **METR-LA** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/liyaguang/DCRNN
- **PEMS-BAY** — total ?; splits: train (-1), val (-1), test (-1); repo https://pems.dot.ca.gov/
- **PeMS04** — total ?; splits: train (-1), val (-1), test (-1); repo https://pems.dot.ca.gov/

## Metrics

- `MAE` **(primary)** — range: other
  - Mean Absolute Error: average of absolute differences between predicted and observed values.
- `RMSE` — range: other
  - Root Mean Square Error: square root of the average of squared differences between predicted and observed values, normalized by standard deviation in the paper's formula.
- `MAPE` — range: percent
  - Mean Absolute Percentage Error: average of absolute percentage differences between predicted and observed values.

## Input / output format

**Input**: Historical traffic sensor readings (speed/occupancy) over a sequence of time steps, optionally with a pre-defined or learned adjacency matrix representing the road network.

**Output**: Predicted traffic flow values (speed or occupancy) for a specified future horizon (15, 30, or 60 minutes).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, sigma=None):
    n = len(y_true)
    mae = sum(abs(y - x) for x, y in zip(y_true, y_pred)) / n
    if sigma is not None:
        rmse = math.sqrt(sum(((y - x) / s) ** 2 for x, y, s in zip(y_true, y_pred, sigma)) / n)
    else:
        rmse = math.sqrt(sum((y - x) ** 2 for x, y in zip(y_true, y_pred)) / n)
    mape = sum(abs(y - x) / x for x, y in zip(y_true, y_pred)) / n
    return mae, rmse, mape
```

## Common pitfalls

- The paper reports RMSE with a non-standard normalization by sigma_i (standard deviation), which differs from the typical RMSE formula.
- Datasets are split by time (e.g., 80% train, 10% val, 10% test) but exact split ratios are not explicitly stated in this section.
- Models are evaluated at multiple forecasting horizons (15, 30, 60 min), and results are not always aggregated into a single score.

## Evidence (verbatim from paper)

> These models are evaluated using metrics mean absolute error (MAE), Root-mean-square error(RMSE) and mean absolute percentage error (MAPE) where $x_{i}$ is the observed value and $y_{i}$ is the predicted value.

## Citation

```bibtex
@misc{li2023gnnspatiotemporal,
  title={Graph Neural Network for spatiotemporal data: methods and applications},
  author={Yun Li et al. (2023)},
  year={2023},
  note={arXiv:2306.00012}
}
```

- arXiv: 2306.00012