Road Traffic Estimation Eval

---
name: road-traffic-estimation-eval
description: Evaluates the ability to estimate traffic flow profiles for unsensed road segments by selecting similar roads based on topological embeddings or generating synthetic data. It probes how well graph-based feature similarity correlates with actual traffic pattern matching and the accuracy of generative models for sensorless traffic estimation. Use when the user wants to benchmark on Madrid Traffic Network, or asks about evaluating this task. Reports RMSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2201.04968
  bibtex_key: manibardo2022graphbased
  confidence: high
---

# road-traffic-estimation-eval

> A Graph-based Methodology for the Sensorless Estimation of Road Traffic Profiles — Manibardo et al. (2022) (arXiv:2201.04968, 2022)

## What this evaluates

Evaluates the ability to estimate traffic flow profiles for unsensed road segments by selecting similar roads based on topological embeddings or generating synthetic data. It probes how well graph-based feature similarity correlates with actual traffic pattern matching and the accuracy of generative models for sensorless traffic estimation.

## Datasets

- **Madrid Traffic Network** — total 55; splits: test (55); repo https://github.com/Eric-L-Manibardo/Road_embedding

## Metrics

- `RMSE` **(primary)** — range: other
  - Root Mean Square Error: sqrt(1/N * sum((y_i - ŷ_i)^2)), where y is the target traffic profile, ŷ is the predicted/generated profile, and N is the number of time samples.
- `nRMSE` — range: percent
  - Normalized RMSE calculated by dividing the RMSE by the mean weekday flow of the target location to enable scale-independent comparison across road segments.

## Input / output format

**Input**: Topological and contextual road features (7 normalized dimensions) for candidate road segments, and target location traffic profiles (time series of vehicle counts over weekdays).

**Output**: Selected candidate road segment's traffic profile or synthetically generated traffic profile (730 days of vehicle counts).

## Scoring recipe

```python
def compute_rmse(y_true, y_pred):
    n = len(y_true)
    return math.sqrt(sum((y - yp)**2 for y, yp in zip(y_true, y_pred)) / n)

def compute_nrmse(y_true, y_pred, mean_weekday_flow):
    return compute_rmse(y_true, y_pred) / mean_weekday_flow

def compute_embedding_similarity(emb1, emb2):
    dist = math.sqrt(sum((a-b)**2 for a, b in zip(emb1, emb2)))
    return (1 - dist / math.sqrt(7)) * 100
```

## Common pitfalls

- nRMSE is normalized specifically by the mean weekday flow of the target location, not by standard deviation or data range as in other literature.
- Embedding similarity is derived from a bounded Euclidean distance (max sqrt(7)) across 7 normalized features, not a standard cosine similarity or correlation metric.
- Statistical significance of generative model differences is evaluated using a Friedman test followed by a Nemenyi post hoc test, not simple pairwise t-tests.

## Evidence (verbatim from paper)

> The Root Mean Square Error (RMSE) has been chosen as error metric, due to its explainability. Obtained deviations can be interpreted as the number of vehicles that might be under/over predicted. Given a desired traffic pattern, the RMSE measures the performance of the selection system: RMSE(y, ŷ) = sqrt(1/N sum(y_i - ŷ_i)^2), where RMSE(y, ŷ) is the RMSE computed over a traffic profile y from a target location and a traffic profile ŷ from a sensed point of the network, and N is the total number of samples that conform the traffic profiles over which the error is computed. Being RMSE a scale dependent error metric, its normalized version or nRMSE is also employed in this section. The chosen approach is to normalize the RMSE by the mean flow of all weekdays from target location.

## Citation

```bibtex
@misc{manibardo2022graphbased,
  title={A Graph-based Methodology for the Sensorless Estimation of Road Traffic Profiles},
  author={Manibardo et al. (2022)},
  year={2022},
  note={arXiv:2201.04968}
}
```

- arXiv: 2201.04968