Generalad Eval

---
name: generalad-eval
description: Evaluates cross-domain anomaly detection capability across semantic, near-distribution, and industrial benchmarks. Probes the model's ability to detect pixel-level defects and semantic novelties using a self-supervised transformer discriminator that attends to distorted features without task-specific tuning. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, Fashion-MNIST, View, Aircraft-FGVC, Stanford Cars, MVTec-AD, MVTec-LOCO, VisA, MPDD, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.12427
  bibtex_key: strater2024generalad
  confidence: high
---

# generalad-eval

> GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features — Sträter et al. (2024) (arXiv:2407.12427, 2024)

## What this evaluates

Evaluates cross-domain anomaly detection capability across semantic, near-distribution, and industrial benchmarks. Probes the model's ability to detect pixel-level defects and semantic novelties using a self-supervised transformer discriminator that attends to distorted features without task-specific tuning.

## Datasets

- **CIFAR-10** — total ?; splits: train (-1), test (-1)
- **CIFAR-100** — total ?; splits: train (-1), test (-1)
- **Fashion-MNIST** — total ?; splits: train (-1), test (-1)
- **View** — total ?; splits: train (-1), test (-1)
- **Aircraft-FGVC** — total ?; splits: train (-1), test (-1)
- **Stanford Cars** — total ?; splits: train (-1), test (-1)
- **MVTec-AD** — total ?; splits: train (-1), test (-1)
- **MVTec-LOCO** — total ?; splits: train (-1), test (-1)
- **VisA** — total ?; splits: train (-1), test (-1)
- **MPDD** — total ?; splits: train (-1), test (-1)

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic curve. Computed separately for image-level binary classification and pixel-level segmentation/localization tasks.

## Input / output format

**Input**: Input images rescaled to 518×518 without augmentations. Processed through a DINOv2 backbone to extract last-layer features, which are normalized. For training, features are distorted via noise injection and patch shuffling to create pseudo-abnormal samples.

**Output**: Anomaly scores at the image level and pixel-level anomaly score maps. Thresholded to produce binary anomaly labels or segmentation masks.

## Scoring recipe

```python
def compute_auroc(scores, labels):
    # scores: anomaly scores (higher = more anomalous)
    # labels: ground truth (0 = normal, 1 = anomalous)
    fpr, tpr, _ = roc_curve(labels, scores)
    return auc(fpr, tpr)
```

## Common pitfalls

- The top-K parameter for feature selection must be dataset-specific: K=1369 for semantic/near datasets (covering all patches) and K=10 for industrial datasets (covering small defects).
- Distortion strategies are task-dependent: 'Noise All Patches' for semantic, 'Noise Random Patches' for industrial, and 'Attention Shuffle' for logical anomalies.
- Input images must be rescaled to 518×518 without any image augmentations to maintain consistency with the DINOv2 backbone.

## Evidence (verbatim from paper)

> We report the AUROC performance for both image-level and pixel-level tasks in tables[1] and[2]. Our experiments encompass three distinct benchmarks. First, we evaluate our approach on four semantic anomaly detection datasets: CIFAR-10, CIFAR-100, Fashion-MNIST, and View. ... Finally, we experiment on four industrial anomaly detection datasets: MVTec-AD, MVTec-LOCO, VisA, and MPDD.

## Citation

```bibtex
@misc{strater2024generalad,
  title={GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features},
  author={Sträter et al. (2024)},
  year={2024},
  note={arXiv:2407.12427}
}
```

- arXiv: 2407.12427