Real Iad Eval

---
name: real-iad-eval
description: Evaluates industrial anomaly detection models under standard unsupervised and fully unsupervised (noisy training) settings. It probes image-level, pixel-level, and multi-view sample-level defect detection capabilities. Use when the user wants to benchmark on Real-IAD, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2403.12580
  bibtex_key: wang2024realiad
  confidence: high
---

# real-iad-eval

> Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection — Wang et al. (2024) (arXiv:2403.12580, 2024)

## What this evaluates

Evaluates industrial anomaly detection models under standard unsupervised and fully unsupervised (noisy training) settings. It probes image-level, pixel-level, and multi-view sample-level defect detection capabilities.

## Datasets

- **Real-IAD** — total 150000; splits: train (-1), test (200)

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic curve, computed separately for image-level and pixel-level anomaly scores.
- `AUPRO` — range: [0, 1]
  - Area Under the Per-Region Overlap curve, measuring pixel-level segmentation accuracy against ground truth masks.

## Input / output format

**Input**: High-resolution industrial product images, optionally provided as multiple views (up to 5 angles) per sample.

**Output**: Per-image and per-pixel anomaly scores, or binary segmentation masks indicating defective regions.

## Scoring recipe

```python
def compute_auroc(y_true, y_scores):
    fpr, tpr, _ = roc_curve(y_true, y_scores)
    return auc(fpr, tpr)

def compute_aupro(pred_masks, gt_masks):
    pros = []
    for pred, gt in zip(pred_masks, gt_masks):
        overlap = compute_intersection_over_union(pred, gt)
        pros.append(overlap)
    return compute_auc(pros)

# Multi-view aggregation: average or max scores across views per sample
sample_scores = aggregate_across_views(image_scores, view_scores)
```

## Common pitfalls

- FUIAD training sets must be constructed with a specific noise ratio (α) rather than using all available anomalies.
- Test set is strictly fixed to 100 normal and 100 anomalous samples (500 images total across views); do not use the full dataset for testing.
- Multi-view performance requires aggregating scores across views to match industrial production line evaluation, not just reporting per-view metrics.

## Evidence (verbatim from paper)

> The AUROC is the most widely used metric for image-level and pixel-level anomaly detection. In addition, a normalized Per-Region Overlap (PRO) between segmentation and ground truth is calculated and the Area Under PRO curve (AUPRO) is also adopted as a pixel-level metric. To create an FUIAD setting, we first need to fix the testing set including normal and anomaly samples. In our experiment, the number of normal and anomaly samples is uniformly set to 100 samples (500 images).

## Citation

```bibtex
@misc{wang2024realiad,
  title={Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection},
  author={Wang et al. (2024)},
  year={2024},
  note={arXiv:2403.12580}
}
```

- arXiv: 2403.12580