Cifar Robustness Eval

---
name: cifar-robustness-eval
description: Evaluates the robustness of adversarially trained neural networks against Projected Gradient Descent (PGD) attacks on CIFAR-10 and CIFAR-100. It measures both clean (natural) classification accuracy and robust accuracy under varying attack strengths (PGD-20 and PGD-100). Use when the user wants to benchmark on CIFAR-10, CIFAR-100, or asks about evaluating this task. Reports PGD-20 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2006.00387
  bibtex_key: xu2020exploring
  confidence: high
---

# cifar-robustness-eval

> Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training — Xu et al. (2020) (arXiv:2006.00387, 2020)

## What this evaluates

Evaluates the robustness of adversarially trained neural networks against Projected Gradient Descent (PGD) attacks on CIFAR-10 and CIFAR-100. It measures both clean (natural) classification accuracy and robust accuracy under varying attack strengths (PGD-20 and PGD-100).

## Datasets

- **CIFAR-10** — total ?; splits: test (-1); HF `cifar10`
- **CIFAR-100** — total ?; splits: test (-1); HF `cifar100`

## Metrics

- `natural accuracy` — range: percent
  - Percentage of correctly classified clean validation images.
- `PGD-20 accuracy` **(primary)** — range: percent
  - Percentage of correctly classified images after applying a 20-step Projected Gradient Descent attack with epsilon=8.
- `PGD-100 accuracy` — range: percent
  - Percentage of correctly classified images after applying a 100-step Projected Gradient Descent attack with epsilon=8.

## Input / output format

**Input**: RGB image from CIFAR-10 or CIFAR-100 dataset.

**Output**: Predicted class label (0-9 for CIFAR-10, 0-99 for CIFAR-100).

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(p == g for p, g in zip(predictions, gold_labels))
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- Attack evaluation parameters (epsilon, step size, iterations) must be strictly controlled; the paper uses epsilon=8 for evaluation but varies training step sizes (epsilon_s=2 or 6).
- Comparisons must account for model capacity differences, as adaptive networks have slightly more parameters than non-adaptive baselines.

## Evidence (verbatim from paper)

> We report validation accuracy on natural images and adversarial images generated using PGD attacks with K=20 iterations and K=100 iterations.

## Citation

```bibtex
@misc{xu2020exploring,
  title={Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training},
  author={Xu et al. (2020)},
  year={2020},
  note={arXiv:2006.00387}
}
```

- arXiv: 2006.00387