Imagenet Top1 Error Eval

---
name: imagenet-top1-error-eval
description: Evaluates the top-1 classification accuracy of a model on the ImageNet dataset. It probes the model's ability to correctly classify images into one of 1000 categories under various training conditions, specifically testing the impact of large minibatch sizes and learning rate scaling strategies on optimization and generalization. Use when the user wants to benchmark on ImageNet, or asks about evaluating this task. Reports top-1 error (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1706.02677
  bibtex_key: goyal2017accurate
  confidence: high
---

# imagenet-top1-error-eval

> Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour — Goyal et al. (2017) (arXiv:1706.02677, 2017)

## What this evaluates

Evaluates the top-1 classification accuracy of a model on the ImageNet dataset. It probes the model's ability to correctly classify images into one of 1000 categories under various training conditions, specifically testing the impact of large minibatch sizes and learning rate scaling strategies on optimization and generalization.

## Datasets

- **ImageNet** — total 1330000; splits: train (1280000), val (50000); HF `imagenet-1k`

## Metrics

- `top-1 error (%)` **(primary)** — range: percent
  - 100 * (1 - (number of correctly classified validation images / total number of validation images)).

## Input / output format

**Input**: 224x224 pixel random crops from augmented images (with horizontal flips), normalized by per-color mean and standard deviation.

**Output**: Class label prediction (top-1) for each validation image.

## Scoring recipe

```python
correct = 0
for img, label in val_dataset:
    pred = model(img)
    if pred == label:
        correct += 1
error_rate = 100 * (1 - correct / len(val_dataset))
```

## Common pitfalls

- Error is computed as the median over the final 5 training epochs, not just the final epoch.
- Results must be averaged over 5 independent runs to account for random variation; single-run results are unreliable.

## Evidence (verbatim from paper)

> The 1000-way ImageNet classification task serves as our main experimental benchmark. Models are trained on the ~1.28 million training images and evaluated by top-1 error on the 50,000 validation images. ... we compute a model’s error rate as the median error of the final 5 epochs. Moreover, we report the mean and standard deviation (std) of the error from 5 independent runs.

## Citation

```bibtex
@misc{goyal2017accurate,
  title={Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour},
  author={Goyal et al. (2017)},
  year={2017},
  note={arXiv:1706.02677}
}
```

- arXiv: 1706.02677