Intersection Scenarios Eval

---
name: intersection-scenarios-eval
description: Evaluates reinforcement learning agents' ability to navigate complex, un-signalized urban intersections under varying traffic conditions. It probes decision-making, collision avoidance, and route completion in dynamic environments with interacting social vehicles. Use when the user wants to benchmark on Intersection Scenarios (RL-CIS), or asks about evaluating this task. Reports Success rate(%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2109.10557
  bibtex_key: liu2021reinforcement
  confidence: high
---

# intersection-scenarios-eval

> A Reinforcement Learning Benchmark for Autonomous Driving in Intersection Scenarios — Liu et al. (2021) (arXiv:2109.10557, 2021)

## What this evaluates

Evaluates reinforcement learning agents' ability to navigate complex, un-signalized urban intersections under varying traffic conditions. It probes decision-making, collision avoidance, and route completion in dynamic environments with interacting social vehicles.

## Datasets

- **Intersection Scenarios (RL-CIS)** — total ?; splits: test (-1); repo https://github.com/liuyuqi123/ComplexUrbanScenarios

## Metrics

- `Success rate(%)` **(primary)** — range: percent
  - Percentage of test episodes where the ego vehicle successfully completes the assigned route without collision or failure. Calculated as (successful runs / total runs) × 100.
- `Average time(s)` — range: other
  - Mean time in seconds taken by the agent to complete the route across all evaluated episodes.

## Input / output format

**Input**: High-resolution CARLA simulator observations including ego-vehicle state, surrounding traffic flow parameters, and route waypoints.

**Output**: Continuous control actions (steering, acceleration, braking) or discrete maneuver commands per simulation step.

## Scoring recipe

```python
def compute_metrics(successes, total_runs, times):
    success_rate = (sum(successes) / total_runs) * 100
    avg_time = sum(times) / len(times)
    return success_rate, avg_time
```

## Common pitfalls

- Deterministic vs. stochastic traffic flow generation significantly impacts results; stochastic uses uniform sampling while deterministic uses logical scenario definitions.
- Rule-based agents have limited input and cannot detect potential conflicts from cross directions, leading to lower safety and success rates compared to RL agents.
- Success rate and average time are calculated per functional scenario (turning left, turning right, going straight), not aggregated globally, so cross-task comparisons require careful normalization.

## Evidence (verbatim from paper)

> We evaluate the TD3 agent and rule-based agents in all five functional scenarios. Since the rule-based agents are poorly performed relatively. The statistics are calculated by the task routes for the rules-based agents. In turning left and turning right experiments, the RL agent reaches a success near  $90\%$ , and exceeds the rules-based agent in both success rate and average time.

## Citation

```bibtex
@misc{liu2021reinforcement,
  title={A Reinforcement Learning Benchmark for Autonomous Driving in Intersection Scenarios},
  author={Liu et al. (2021)},
  year={2021},
  note={arXiv:2109.10557}
}
```

- arXiv: 2109.10557