Daad X Eval

---
name: daad-x-eval
description: Evaluates a model's ability to predict driver maneuvers and generate hierarchical, human-understandable explanations from ego-centric driving videos. It probes spatio-temporal feature disentanglement and the impact of gaze modality on interpretability. Use when the user wants to benchmark on DAAD-X, or asks about evaluating this task. Reports Acc.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2510.09200
  bibtex_key: karuppasamy2025towards
  confidence: high
---

# daad-x-eval

> Towards Safer and Understandable Driver Intention Prediction — Karuppasamy et al. (2025) (arXiv:2510.09200, 2025)

## What this evaluates

Evaluates a model's ability to predict driver maneuvers and generate hierarchical, human-understandable explanations from ego-centric driving videos. It probes spatio-temporal feature disentanglement and the impact of gaze modality on interpretability.

## Datasets

- **DAAD-X** — total ?; splits: test (-1)

## Metrics

- `Acc` **(primary)** — range: [0, 1]
  - Proportion of correctly predicted action or explanation labels out of the total number of instances.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall for action prediction.
- `F1(mac)` — range: [0, 1]
  - Macro-averaged F1 score across all explanation classes, treating each class equally regardless of frequency.
- `F1(mic)` — range: [0, 1]
  - Micro-averaged F1 score across all explanation classes, calculated by aggregating contributions of all classes before computing precision and recall.

## Input / output format

**Input**: Ego-centric driving video frames (optionally with gaze modality provided as no gaze, overlaid, or cropped circular regions). Processed through pre-trained backbones (I3D, VideoMAE, MViTv2).

**Output**: Predicted action label and multi-label explanation predictions (binary activation masks for each explanation class).

## Scoring recipe

```python
def compute_metrics(preds, gold):
    acc = (preds == gold).mean()
    tp = (preds & gold).sum()
    fp = (preds & ~gold).sum()
    fn = (~preds & gold).sum()
    prec = tp / (tp + fp + 1e-8)
    rec = tp / (tp + fn + 1e-8)
    f1 = 2 * prec * rec / (prec + rec + 1e-8)
    return acc, f1
```

## Common pitfalls

- Gaze modality handling significantly impacts results; cropped gaze regions outperform overlaid or absent gaze, but excessive crop sizes degrade performance.
- Temporal cues are critical for explanation generation; frame reshuffling disproportionately hurts transformer-based models compared to CNNs, which rely more on spatial features.
- Multi-label t-SNE visualization requires computing anchor points by averaging features per explanation class, rather than applying t-SNE directly to multi-label vectors.

## Evidence (verbatim from paper)

> Table 2: Evaluation on DAAD-X dataset: Evaluated baselines with (wB) and without (woB) bottleneck. Here, LTM indicates Learnable Token Merging.

| Model | Action | | ego-vehicle eXplanation | | | |
| --- | --- | --- | --- | --- | --- | --- |
| | Acc | $F_{1}$ | Acc | $F_{1}$ | $F_{1}(mac)$ | $F_{1}(mic)$ |

## Citation

```bibtex
@misc{karuppasamy2025towards,
  title={Towards Safer and Understandable Driver Intention Prediction},
  author={Karuppasamy et al. (2025)},
  year={2025},
  note={arXiv:2510.09200}
}
```

- arXiv: 2510.09200