Fractal Pretraining Eval

---
name: fractal-pretraining-eval
description: Evaluates the downstream transfer capability of fractal-based pre-trained visual representations on fine-grained image classification and medical image segmentation tasks. It measures how effectively synthetic Iterated Function System (IFS) pre-training captures transferable features compared to training from scratch or using ImageNet/FractalDB pre-training. Use when the user wants to benchmark on CUB-2011, Stanford Cars, Stanford Dogs, FGVC Aircraft, CIFAR-100, GlaS, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2110.03091
  bibtex_key: anderson2021improvingfractal
  confidence: high
---

# fractal-pretraining-eval

> Improving Fractal Pre-training — Anderson et al. (2021) (arXiv:2110.03091, 2021)

## What this evaluates

Evaluates the downstream transfer capability of fractal-based pre-trained visual representations on fine-grained image classification and medical image segmentation tasks. It measures how effectively synthetic Iterated Function System (IFS) pre-training captures transferable features compared to training from scratch or using ImageNet/FractalDB pre-training.

## Datasets

- **CUB-2011** — total ?; splits: test (-1)
- **Stanford Cars** — total ?; splits: test (-1)
- **Stanford Dogs** — total ?; splits: test (-1)
- **FGVC Aircraft** — total ?; splits: test (-1)
- **CIFAR-100** — total ?; splits: test (-1)
- **GlaS** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Top-1 classification accuracy, calculated as the fraction of correctly predicted class labels over the total number of test instances. For segmentation tasks, performance is reported qualitatively alongside classification metrics.

## Input / output format

**Input**: 224×224 RGB images for classification tasks; corresponding pixel-wise segmentation masks for the GlaS dataset.

**Output**: Discrete class labels for classification tasks; binary or multi-class segmentation masks for the GlaS dataset.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    if len(predictions) != len(gold_labels):
        raise ValueError('Prediction and gold label lengths must match')
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- The conference version of the paper contained a code bug claiming 50,000 IFS codes were used when only 1,000 were actually sampled, invalidating specific ablation results about system count vs. parameter augmentation.
- Fine-tuning hyperparameters differ significantly between tasks (150 epochs/batch 96 for classification vs. 90 epochs/batch 8 for segmentation), so cross-task performance comparisons require strict adherence to these settings.
- Evaluating synthetic fractal pre-training on natural image benchmarks (e.g., CIFAR-100, CUB) introduces a domain gap that may not reflect performance on out-of-distribution or medical imaging data.

## Evidence (verbatim from paper)

> We evaluate the effectiveness of the pre-trained representations by fine-tuning on several different tasks. For image classification, we use CUB-2011, Stanford Cars, Stanford Dogs, FGVC Aircraft and CIFAR-100. We also fine-tune models for medical image segmentation on the GlaS dataset. ... In fact, multi-instance prediction models can provide more than 90% of the accuracy achieved by ImageNet pre-training—and in some cases, such as for Stanford Cars, the model obtains over 98% of the ImageNet performance.

## Citation

```bibtex
@misc{anderson2021improvingfractal,
  title={Improving Fractal Pre-training},
  author={Anderson et al. (2021)},
  year={2021},
  note={arXiv:2110.03091}
}
```

- arXiv: 2110.03091