Urban Pathfinding Eval

---
name: urban-pathfinding-eval
description: Evaluates real-time urban pathfinding algorithms under dynamic traffic and weather conditions. It measures how well traditional graph search methods and deep learning models predict optimal routes and minimize travel time in a simulated Berlin city environment. Use when the user wants to benchmark on Berlin Urban Simulation, or asks about evaluating this task. Reports Average Travel Time (s).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.05044
  bibtex_key: abo2024deep
  confidence: medium
---

# urban-pathfinding-eval

> Deep Heuristic Learning for Real-Time Urban Pathfinding — Abo El-Ela and Hamdi (2024) (arXiv:2411.05044, 2024)

## What this evaluates

Evaluates real-time urban pathfinding algorithms under dynamic traffic and weather conditions. It measures how well traditional graph search methods and deep learning models predict optimal routes and minimize travel time in a simulated Berlin city environment.

## Datasets

- **Berlin Urban Simulation** — total ?; splits: test (-1)

## Metrics

- `Average Travel Time (s)` **(primary)** — range: other
  - Mean total time in seconds to traverse the predicted route across all test scenarios. Lower values indicate better performance.
- `Improvement (%)` — range: percent
  - Percentage reduction in average travel time relative to the baseline Autoencoder model, calculated as ((Baseline - Algorithm) / Baseline) * 100.
- `F1 scores` — range: [0, 1]
  - Harmonic mean of Precision and Recall for path segment prediction.

## Input / output format

**Input**: Graph-based urban road network data augmented with real-time traffic congestion levels and weather conditions, provided as sequential or tabular features for neural models and as node/edge weights for search algorithms.

**Output**: Predicted optimal route sequence and estimated travel time, or binary classification of optimal path segments.

## Scoring recipe

```python
def compute_metrics(predictions, gold, baseline_time=7260.0):
    travel_times = [pred['time'] for pred in predictions]
    avg_time = sum(travel_times) / len(travel_times)
    improvement = ((baseline_time - avg_time) / baseline_time) * 100
    tp = sum(1 for p, g in zip(predictions, gold) if p['segment'] == 1 and g == 1)
    fp = sum(1 for p, g in zip(predictions, gold) if p['segment'] == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold) if p['segment'] == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return {'avg_time': avg_time, 'improvement': improvement, 'f1': f1}
```

## Common pitfalls

- Baseline comparison uses Autoencoder as the 0% improvement reference rather than a standard algorithm like Dijkstra or A*, which may skew perceived gains.
- Reported accuracy/precision/F1 likely refer to path segment classification rather than end-to-end route optimality, which can mask suboptimal global routing.
- Computational overhead and inference latency of neural models are not included in the travel time metric, potentially overestimating real-world viability.

## Evidence (verbatim from paper)

> The performance metrics, as shown in Table [II], further demonstrate the advantages of deep learning models, especially MLP and Transformer, which achieved better accuracy, precision, and F1 scores than traditional algorithms and heuristic A*.

## Citation

```bibtex
@misc{abo2024deep,
  title={Deep Heuristic Learning for Real-Time Urban Pathfinding},
  author={Abo El-Ela and Hamdi (2024)},
  year={2024},
  note={arXiv:2411.05044}
}
```

- arXiv: 2411.05044