Surveillance Anomaly Detection Eval

---
name: surveillance-anomaly-detection-eval
description: Evaluates a model's ability to detect and temporally localize anomalous events in long, untrimmed surveillance videos using only video-level labels. It probes the model's robustness to high intra-class variation, ambiguous normal-anomalous boundaries, and varying lighting/occlusion conditions. Use when the user wants to benchmark on Surveillance Anomaly Dataset, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1801.04264
  bibtex_key: sultani2018realworld
  confidence: high
---

# surveillance-anomaly-detection-eval

> Real-world Anomaly Detection in Surveillance Videos — Sultani et al. (2018) (arXiv:1801.04264, 2018)

## What this evaluates

Evaluates a model's ability to detect and temporally localize anomalous events in long, untrimmed surveillance videos using only video-level labels. It probes the model's robustness to high intra-class variation, ambiguous normal-anomalous boundaries, and varying lighting/occlusion conditions.

## Datasets

- **Surveillance Anomaly Dataset** — total 1900; splits: train (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: percent
  - Area under the frame-based Receiver Operating Characteristic (ROC) curve. Computed by plotting the true positive rate against the false positive rate at various threshold settings on frame-level anomaly scores.

## Input / output format

**Input**: Untrimmed surveillance video frames (resized to 240x320, 30 fps), processed in 16-frame clips and averaged into 32 non-overlapping temporal segments per video.

**Output**: A continuous anomaly score per temporal segment (or frame), where higher values indicate a higher likelihood of containing an anomaly.

## Scoring recipe

```python
scores = model.predict_segments(video)
gt = get_frame_level_ground_truth(video)
roc = compute_roc_curve(scores, gt)
auc = integrate(roc)
return auc * 100
```

## Common pitfalls

- Using Equal Error Rate (EER) instead of AUC, which fails to correctly measure performance when anomalies occupy only a small fraction of long videos.
- Applying standard action recognition (whole-video binary classification) instead of temporal localization, as untrimmed videos contain mostly normal content and high intra-class variation.
- Assuming segment-level annotations are available during training; the protocol is weakly supervised, relying solely on video-level normal/anomalous labels.

## Evidence (verbatim from paper)

> Following previous works on anomaly detection [[27]], we use frame based receiver operating characteristic (ROC) curve and corresponding area under the curve (AUC) to evaluate the performance of our method. We do not use equal error rate (EER) [[27]] as it does not measure anomaly correctly, specifically if only a small portion of a long video contains anomalous behavior.

## Citation

```bibtex
@misc{sultani2018realworld,
  title={Real-world Anomaly Detection in Surveillance Videos},
  author={Sultani et al. (2018)},
  year={2018},
  note={arXiv:1801.04264}
}
```

- arXiv: 1801.04264