Dinov2 Linear Eval

---
name: dinov2-linear-eval
description: Evaluates the quality of frozen self-supervised visual features by training a simple linear classifier on top of them across diverse image and video understanding tasks, probing generalization, robustness, and instance-level recognition capabilities. Use when the user wants to benchmark on ImageNet-1k, ImageNet-V2, ImageNet-ReaL, iNaturalist, Places205, UCF-101, Kinetics-400, Something-Something v2, Oxford/Paris, ImageNet-A, ImageNet-R, ImageNet-C, Sketch, or asks about evaluating this task. Reports Top-1 accuracy (linear evaluation).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2304.07193
  bibtex_key: oquab2023dinov2
  confidence: high
---

# dinov2-linear-eval

> DINOv2: Learning Robust Visual Features without Supervision — Maxime Oquab et al. (2023) (arXiv:2304.07193, 2023)

## What this evaluates

Evaluates the quality of frozen self-supervised visual features by training a simple linear classifier on top of them across diverse image and video understanding tasks, probing generalization, robustness, and instance-level recognition capabilities.

## Datasets

- **ImageNet-1k** — total 1281167; splits: train (1281167), val (50000); HF `imagenet-1k`
- **ImageNet-V2** — total 10000; splits: test (10000); HF `imagenet-v2`
- **ImageNet-ReaL** — total 10000; splits: test (10000)
- **iNaturalist** — total ?; splits: val (-1); HF `iNaturalist`
- **Places205** — total ?; splits: test (-1); HF `places205`
- **UCF-101** — total 13320; splits: test (3780); HF `ucf101`
- **Kinetics-400** — total ?; splits: test (-1); HF `kinetics400`
- **Something-Something v2** — total ?; splits: test (-1); HF `something-something-v2`
- **Oxford/Paris** — total ?; splits: test (-1)
- **ImageNet-A** — total 7500; splits: test (7500); HF `imagenet-a`
- **ImageNet-R** — total 30000; splits: test (30000); HF `imagenet-r`
- **ImageNet-C** — total 30000; splits: test (30000); HF `imagenet-c`
- **Sketch** — total 10000; splits: test (10000); HF `sketch`

## Metrics

- `Top-1 accuracy (linear evaluation)` **(primary)** — range: percent
  - Percentage of correctly classified instances when training a linear classifier (logistic regression or linear layer) on frozen backbone features. Computed on the validation or test set.
- `kNN accuracy` — range: percent
  - Accuracy computed using a k-Nearest Neighbors classifier on frozen features, typically evaluated on the validation set.
- `mAP` — range: percent
  - Mean Average Precision used for image retrieval benchmarks (e.g., Oxford/Paris).

## Input / output format

**Input**: Images (and videos sampled as evenly spaced frames) fed into a frozen Vision Transformer backbone to extract feature embeddings.

**Output**: Class predictions from a linear probe trained on frozen features, or feature vectors for retrieval tasks.

## Scoring recipe

```python
def linear_eval(frozen_features, labels):
    # frozen_features: shape (N, D)
    # labels: shape (N,)
    model = LinearClassifier()
    model.train(frozen_features, labels)
    preds = model.predict(frozen_features)
    return accuracy(preds, labels) * 100
```

## Common pitfalls

- Using full end-to-end fine-tuning instead of the specified linear probing protocol, which invalidates the comparison to other SSL methods.
- Not using frozen backbone weights; the evaluation explicitly requires features to remain frozen during the downstream task.
- Confusing linear evaluation with kNN evaluation; the paper reports both but uses linear evaluation as the primary headline metric.

## Evidence (verbatim from paper)

> We evaluate the quality of features by training a simple classifier over a frozen backbone, and do not perform finetuning of the backbone weights. Following previous work, we use a linear model for simplicity, ensuring a reproducible evaluation... We report Top-1 accuracy on the validation set for publicly available models trained on public or private data, and with or without text supervision.

## Citation

```bibtex
@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Maxime Oquab et al. (2023)},
  year={2023},
  note={arXiv:2304.07193}
}
```

- arXiv: 2304.07193