Sfld Ai Gen Image Detection Eval

---
name: sfld-ai-gen-image-detection-eval
description: Evaluates AI-generated image detectors on their ability to generalize across diverse generative models (GANs, diffusion) and resist content bias. It probes robustness using conventional benchmarks, a new content-preserving benchmark (TwinSynths), and low-level vision/perceptual benchmarks to measure how well models rely on texture vs. semantic artifacts. Use when the user wants to benchmark on Conventional benchmark, TwinSynths, Low-level vision and perceptual benchmarks, or asks about evaluating this task. Reports AP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2502.17105
  bibtex_key: gye2025sfld
  confidence: high
---

# sfld-ai-gen-image-detection-eval

> SFLD: Reducing the content bias for AI-generated Image Detection — Gye et al. (2025) (arXiv:2502.17105, 2025)

## What this evaluates

Evaluates AI-generated image detectors on their ability to generalize across diverse generative models (GANs, diffusion) and resist content bias. It probes robustness using conventional benchmarks, a new content-preserving benchmark (TwinSynths), and low-level vision/perceptual benchmarks to measure how well models rely on texture vs. semantic artifacts.

## Datasets

- **Conventional benchmark** — total ?; splits: test (-1)
- **TwinSynths** — total ?; splits: test (-1)
- **Low-level vision and perceptual benchmarks** — total ?; splits: test (-1)

## Metrics

- `AP` **(primary)** — range: percent
  - Average Precision; a threshold-independent metric computed as the area under the precision-recall curve across all decision thresholds.
- `Acc.` — range: percent
  - Classification accuracy calculated by applying a fixed decision threshold of 0.5 to model outputs and comparing predicted binary labels with ground truth.

## Input / output format

**Input**: RGB images (real and AI-generated) processed according to each detector's standard input pipeline.

**Output**: Binary classification prediction (real vs. synthetic) or continuous probability score for each image.

## Scoring recipe

```python
def compute_ap(scores, labels):
    precisions, recalls, _ = precision_recall_curve(labels, scores)
    return auc(recalls, precisions) * 100

def compute_acc(scores, labels):
    preds = (scores >= 0.5).astype(int)
    return np.mean(preds == labels) * 100
```

## Common pitfalls

- AP is threshold-independent while Acc. uses a fixed 0.5 threshold; comparing them directly without noting the threshold difference is misleading.
- High scores on conventional benchmarks often reflect content bias rather than true generalization, as shown by significant performance drops on TwinSynths.
- All baselines were trained exclusively on a 20-class ProGAN subset, so evaluation tests zero-shot generalization to unseen generators and post-processing methods.

## Evidence (verbatim from paper)

> We assess the performances of the detection models by average precision score (AP) and classification accuracy (Acc.), following previous works[[49], [32], [46]]. The AP metric is not dependent on the threshold value, whereas the Acc. is calculated with a fixed threshold of 0.5 across all generation models.

## Citation

```bibtex
@misc{gye2025sfld,
  title={SFLD: Reducing the content bias for AI-generated Image Detection},
  author={Gye et al. (2025)},
  year={2025},
  note={arXiv:2502.17105}
}
```

- arXiv: 2502.17105