Medirl Driver Attention Eval

---
name: medirl-driver-attention-eval
description: Predicts driver visual attention (fixation locations) in critical driving scenarios by modeling goal-directed attention sequences. It evaluates how well a model's predicted saliency map matches human gaze patterns across spatial and spatiotemporal cues. Use when the user wants to benchmark on DR(eye)VE, BDD-A, DADA-2000, EyeCar, or asks about evaluating this task. Reports CC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1912.07773
  bibtex_key: baee2019medirl
  confidence: high
---

# medirl-driver-attention-eval

> MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning — Baee et al. (2019) (arXiv:1912.07773, 2019)

## What this evaluates

Predicts driver visual attention (fixation locations) in critical driving scenarios by modeling goal-directed attention sequences. It evaluates how well a model's predicted saliency map matches human gaze patterns across spatial and spatiotemporal cues.

## Datasets

- **DR(eye)VE** — total 400; splits: train (280), val (40), test (80)
- **BDD-A** — total 1350; splits: train (945), val (135), test (270)
- **DADA-2000** — total 534; splits: train (374), val (53), test (107)
- **EyeCar** — total ?; splits: test (-1); repo https://github.com/soniabaee/MEDIRL-EyeCar

## Metrics

- `CC` **(primary)** — range: [0, 1]
  - Pearson Correlation Coefficient between flattened predicted and ground truth saliency maps. Higher values indicate better alignment with human fixation distributions.
- `KLD` — range: other
  - Kullback-Leibler Divergence between predicted and ground truth fixation distributions. Lower values indicate closer match to human attention patterns.
- `s-AUC` — range: [0, 1]
  - shuffled Area under the ROC curve, computed by comparing fixation locations against a shuffled background of non-fixated pixels. Higher values indicate better discrimination, penalizing central prediction bias.

## Input / output format

**Input**: Sequences of 6 video frames resized to 144×256 and normalized. Frames are encoded using pre-trained backbones (HRNetV2, MaskTrack-RCNN, Monodepth2, VPGNet) to extract visual features including target, non-target, distance, lane, lead vehicle, driving task, and speed.

**Output**: A spatial probability map (saliency map) over the frame, generated by 7 softmax units representing fixation likelihood.

## Scoring recipe

```python
def compute_metrics(pred_map, gt_map):
    pred = pred_map.flatten()
    gt = gt_map.flatten()
    cc = pearsonr(pred, gt)
    kld = kl_divergence(gt, pred)
    s_auc = shuffled_auc(gt, pred)
    return {'CC': cc, 'KLD': kld, 's-AUC': s_auc}
```

## Common pitfalls

- Driver attention naturally exhibits a strong center bias; models must be evaluated with s-AUC to avoid inflated scores from predicting central fixations.
- Training data is heavily filtered to exclude frames with >40% focus on irrelevant objects and to ensure KLD > 0.89 from average maps, which may limit generalization to routine driving.
- Metrics are computed on event-level sequences rather than isolated frames, requiring temporal consistency in predictions.

## Evidence (verbatim from paper)

> To evaluate attention prediction, we use location-based and distribution-based saliency metrics: KLD, shuffled Area under the ROC curve (s-AUC), and Correlation Coefficient (CC). We report s-AUC since it penalizes models with more central prediction.

## Citation

```bibtex
@misc{baee2019medirl,
  title={MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning},
  author={Baee et al. (2019)},
  year={2019},
  note={arXiv:1912.07773}
}
```

- arXiv: 1912.07773