Vad Anticipation Eval

---
name: vad-anticipation-eval
description: Evaluates a model's ability to detect anomalous events in surveillance videos and anticipate their occurrence in future frames. It specifically probes scene-dependent anomaly recognition and multi-step temporal anticipation. Use when the user wants to benchmark on ShanghaiTech, CUHK Avenue, IITB Corridor, NWPU Campus, ShanghaiTech-sd, or asks about evaluating this task. Reports AUC (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2305.13611
  bibtex_key: cao2023nwpu
  confidence: high
---

# vad-anticipation-eval

> A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and Anticipation — Cao et al. (2023) (arXiv:2305.13611, 2023)

## What this evaluates

Evaluates a model's ability to detect anomalous events in surveillance videos and anticipate their occurrence in future frames. It specifically probes scene-dependent anomaly recognition and multi-step temporal anticipation.

## Datasets

- **ShanghaiTech** — total ?; splits: test (-1)
- **CUHK Avenue** — total ?; splits: test (-1)
- **IITB Corridor** — total ?; splits: test (-1)
- **NWPU Campus** — total ?; splits: test (-1)
- **ShanghaiTech-sd** — total ?; splits: test (-1)

## Metrics

- `AUC (%)` **(primary)** — range: percent
  - Area under the Receiver Operating Characteristic (ROC) curve computed over concatenated frame-level anomaly scores across the entire dataset.

## Input / output format

**Input**: 256×256 pixel crops centered on objects detected by ByteTrack, processed in sequences of 8 frames.

**Output**: Frame-level anomaly scores for detection (current frame) and anticipation (future frames up to 3 seconds ahead).

## Scoring recipe

```python
# 1. Compute frame-level anomaly scores (e.g., reconstruction error)
frame_scores = model.predict(video_crops)
# 2. Concatenate all frame scores across the dataset
all_scores = np.concatenate(frame_scores)
# 3. Compute ROC AUC against ground truth frame labels
auc_percent = roc_auc_score(ground_truth_labels, all_scores) * 100
```

## Common pitfalls

- Computing AUC per video instead of concatenating all frames across the dataset first.
- Failing to account for scene-dependent anomalies (e.g., treating 'cycling' as abnormal in all scenes rather than only specific ones).
- Using inaccurate object tracking on low-resolution datasets (like CUHK Avenue), which degrades crop quality and scores.

## Evidence (verbatim from paper)

> We use the area under the curve (AUC) of receiver operating characteristic (ROC) to evaluate the performance for both VAD and VAA. Note that we concatenate all the frames in a dataset and then compute the overall frame-level AUC, which is widely adopted.

## Citation

```bibtex
@misc{cao2023nwpu,
  title={A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and Anticipation},
  author={Cao et al. (2023)},
  year={2023},
  note={arXiv:2305.13611}
}
```

- arXiv: 2305.13611