Imagenet32 Eval

---
name: imagenet32-eval
description: Evaluates image classification performance on downsampled variants of ImageNet to test whether lower-resolution datasets can serve as reliable proxies for full-resolution ImageNet in hyperparameter tuning and architecture search. It probes the stability of optimal hyperparameters and model performance across different spatial resolutions while maintaining the original dataset's class structure and image count. Use when the user wants to benchmark on ImageNet32x32, ImageNet64x64, ImageNet16x16, or asks about evaluating this task. Reports validation error rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1707.08819
  bibtex_key: [REDACTED]
  confidence: medium
---

# imagenet32-eval

> A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets — Chrabaszcz et al. (2017) (arXiv:1707.08819, 2017)

## What this evaluates

Evaluates image classification performance on downsampled variants of ImageNet to test whether lower-resolution datasets can serve as reliable proxies for full-resolution ImageNet in hyperparameter tuning and architecture search. It probes the stability of optimal hyperparameters and model performance across different spatial resolutions while maintaining the original dataset's class structure and image count.

## Datasets

- **ImageNet32x32** — total ?; splits: train (-1), val (-1); repo https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
- **ImageNet64x64** — total ?; splits: train (-1), val (-1); repo https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
- **ImageNet16x16** — total ?; splits: train (-1), val (-1); repo https://github.com/PatrykChrabaszcz/Imagenet32_Scripts

## Metrics

- `validation error rate` **(primary)** — range: percent
  - Fraction of misclassified images in the validation set. Computed as (number of incorrect predictions) / (total validation samples). Reported after 31 epochs of training.

## Input / output format

**Input**: RGB images resized to 32×32, 64×64, or 16×16 pixels. During training, images are augmented with horizontal flips and random shifts of up to 4 pixels horizontally and vertically.

**Output**: Predicted class label from the 1000 ImageNet categories.

## Scoring recipe

```python
def compute_validation_error_rate(predictions, gold_labels):
    total = len(gold_labels)
    errors = sum(1 for p, g in zip(predictions, gold_labels) if p != g)
    return errors / total
```

## Common pitfalls

- Network architecture is explicitly modified per resolution (adding or removing residual stacks), so performance differences may reflect architectural adaptations rather than pure dataset difficulty.
- The learning rate schedule is fixed (dropped by factor 5.0 every 10 epochs) regardless of resolution, which may not be optimal for all variants and can skew error rate comparisons.
- Data augmentation is limited to horizontal flips and small random shifts, unlike standard ImageNet's random crops and flips, potentially underestimating true classification performance.

## Evidence (verbatim from paper)

> Throughout, we show validation error rates obtained after training for 31 epochs (right after the last drop of the learning rate).

## Citation

```bibtex
@misc{chrabaszcz2017imagenet32,
  title={A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets},
  author={Chrabaszcz et al. (2017)},
  year={2017},
  note={arXiv:1707.08819}
}
```

- arXiv: 1707.08819