Lane Level Traffic Prediction Eval

---
name: lane-level-traffic-prediction-eval
description: Evaluates models' ability to predict lane-level traffic speed and flow by modeling spatio-temporal dependencies on graph-structured lane networks. It tests performance across both regular and irregular lane configurations, emphasizing both predictive accuracy and training efficiency. Use when the user wants to benchmark on PeMS, PeMSF, HuaNan, or asks about evaluating this task. Reports MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2403.14941
  bibtex_key: li2024unifying
  confidence: high
---

# lane-level-traffic-prediction-eval

> Unifying Lane-Level Traffic Prediction from a Graph Structural Perspective: Benchmark and Baseline — Shuhao Li et al. (arXiv:2403.14941, 2024)

## What this evaluates

Evaluates models' ability to predict lane-level traffic speed and flow by modeling spatio-temporal dependencies on graph-structured lane networks. It tests performance across both regular and irregular lane configurations, emphasizing both predictive accuracy and training efficiency.

## Datasets

- **PeMS** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/ShuhaoLii/TITS24LaneLevel-Traffic-Benchmark
- **PeMSF** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/ShuhaoLii/TITS24LaneLevel-Traffic-Benchmark
- **HuaNan** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/ShuhaoLii/TITS24LaneLevel-Traffic-Benchmark

## Metrics

- `MAE` **(primary)** — range: [0, ∞)
  - MAE = (1/z) * Σ_{t=1}^z (1/N) * Σ_{j=1}^J (1/I) * Σ_{i=1}^I |y_t^{l_{i,j}} - ŷ_t^{l_{i,j}}|, where z is prediction horizon, N is number of time steps, J is lanes, I is segments.
- `RMSE` — range: [0, ∞)
  - RMSE = sqrt((1/z) * Σ_{t=1}^z (1/N) * Σ_{j=1}^J (1/I) * Σ_{i=1}^I (y_t^{l_{i,j}} - ŷ_t^{l_{i,j}})^2).
- `MAPE` — range: percent
  - MAPE = (1/z) * Σ_{t=1}^z (1/N) * Σ_{j=1}^J (1/I) * Σ_{i=1}^I |y_t^{l_{i,j}} - ŷ_t^{l_{i,j}}| / y_t^{l_{i,j}}.
- `Cost` — range: seconds
  - Training expenditure metric calculated as the time required for each iteration of model training multiplied by 100 (or 10^-2 per table), unit in seconds.

## Input / output format

**Input**: Time-series speed and flow measurements from traffic sensors across multiple lanes, organized according to the physical lane topology (graph structure).

**Output**: Predicted speed and flow values for each lane segment at specified future time horizons (e.g., 3, 6, 12 steps).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, z, N, J, I):
    # y_true, y_pred shape: (z, N, J, I)
    mae = np.mean(np.abs(y_true - y_pred))
    rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
    mape = np.mean(np.abs((y_true - y_pred) / y_true)) * 100
    return mae, rmse, mape
```

## Common pitfalls

- Models must handle irregular lane counts (e.g., entrance lanes adding a 6th lane) without breaking graph topology assumptions.
- Training cost is evaluated as a first-class metric (time per iteration * 10^-2), not just accuracy; ignoring efficiency misrepresents practical utility.
- Missing data is imputed using adjacent time slot means, which may smooth out sudden traffic spikes and affect MAPE/RMSE.

## Evidence (verbatim from paper)

> In the field of traffic prediction, particularly at the lane level, a model’s accuracy is typically assessed using three key metrics: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). However, beyond accuracy, the practicality of a model also hinges on the duration of its training. This is especially crucial in lane-level traffic prediction scenarios where rapid response and real-time updates are essential. Therefore, focusing solely on predictive accuracy while overlooking training time could limit the model’s performance in practical applications. Acknowledging this, we have included training expenditure as an evaluation metric, aiming to ensure that models consider both training time and predictive accuracy. The training Cost metric is calculated as the time required for each iteration of model training multiplied by 100, with the unit being seconds.

## Citation

```bibtex
@misc{li2024unifying,
  title={Unifying Lane-Level Traffic Prediction from a Graph Structural Perspective: Benchmark and Baseline},
  author={Shuhao Li et al.},
  year={2024},
  note={arXiv:2403.14941}
}
```

- arXiv: 2403.14941