Fractaldb Pretrain Eval

---
name: fractaldb-pretrain-eval
description: Evaluates the effectiveness of pre-training convolutional neural networks on automatically generated fractal image datasets (FractalDB) compared to natural image pre-training and self-supervised learning, measuring downstream classification accuracy on standard benchmarks. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, ImageNet-100, Places-30, ImageNet-1k, Places-365, Pascal VOC 2012, Omniglot, or asks about evaluating this task. Reports classification accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2101.08515
  bibtex_key: kataoka2021pretraining
  confidence: high
---

# fractaldb-pretrain-eval

> Pre-training without Natural Images — Kataoka et al. (2021) (arXiv:2101.08515, 2021)

## What this evaluates

Evaluates the effectiveness of pre-training convolutional neural networks on automatically generated fractal image datasets (FractalDB) compared to natural image pre-training and self-supervised learning, measuring downstream classification accuracy on standard benchmarks.

## Datasets

- **CIFAR-10** — total ?; splits: train (-1), test (-1)
- **CIFAR-100** — total ?; splits: train (-1), test (-1)
- **ImageNet-100** — total ?; splits: train (-1), test (-1)
- **Places-30** — total ?; splits: train (-1), test (-1)
- **ImageNet-1k** — total ?; splits: train (-1), test (-1)
- **Places-365** — total ?; splits: train (-1), test (-1)
- **Pascal VOC 2012** — total ?; splits: train (-1), test (-1)
- **Omniglot** — total ?; splits: train (-1), test (-1)

## Metrics

- `classification accuracy` **(primary)** — range: percent
  - Percentage of correctly classified instances out of the total number of instances in the evaluation set.

## Input / output format

**Input**: 224x224 pixel RGB images (cropped from 256x256 inputs) with class labels.

**Output**: Predicted class label (integer index or category name) for each image.

## Scoring recipe

```python
def compute_classification_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- The paper uses reduced-category subsets (ImageNet-100, Places-30) for hyperparameter exploration, which can overestimate performance compared to full benchmarks.
- Fine-tuning hyperparameters (batch size 256, LR 0.01 with step decay at epochs 30/60) are fixed across all datasets, deviating from standard dataset-specific fine-tuning protocols.
- Pre-training uses 'formula-supervision' (mathematically generated labels) rather than human annotations, making direct accuracy comparisons with supervised baselines sensitive to label quality and category alignment.

## Evidence (verbatim from paper)

> Table 6: Classification accuracies of the Ours (FractalDB-1k/10k), Scratch, DeepCluster-10k (DC-10k), ImageNet-100/1k and Places-30/365 pre-trained models on representative pre-training datasets. We show the types of pre-trained image (Pre-train Img; which includes {Natural Image (Natural), Formula-driven Image (Formula)}) and Supervision types (Type; which includes {Self-supervision, Supervision, Formula-supervision}).

## Citation

```bibtex
@misc{kataoka2021pretraining,
  title={Pre-training without Natural Images},
  author={Kataoka et al. (2021)},
  year={2021},
  note={arXiv:2101.08515}
}
```

- arXiv: 2101.08515