Dinov3 Medical Vision Eval

---
name: dinov3-medical-vision-eval
description: Evaluates the cross-domain generalization and scaling behavior of a natural-image pre-trained vision transformer (DINOv3) across diverse medical imaging modalities, including 2D/3D classification and segmentation tasks. Use when the user wants to benchmark on NIH-14, RSNA-Pneumonia, Camelyon16, Camelyon17, BCNB, Kvasir-Capsule, AutoLaparo, EndoVis18, EDD 2020, CT-RATE, Medical Segmentation Decathlon (MSD), CREMI, AC3/4, AutoPET-II, HECKTOR 2022, or asks about evaluating this task. Reports AUC, Dice score, IoU (Intersection over Union).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.06467
  bibtex_key: liu2025dino3medical
  confidence: high
---

# dinov3-medical-vision-eval

> Does DINOv3 Set a New Medical Vision Standard? — Liu et al. (2025) (arXiv:2509.06467, 2025)

## What this evaluates

Evaluates the cross-domain generalization and scaling behavior of a natural-image pre-trained vision transformer (DINOv3) across diverse medical imaging modalities, including 2D/3D classification and segmentation tasks.

## Datasets

- **NIH-14** — total ?; splits: test (-1)
- **RSNA-Pneumonia** — total ?; splits: test (-1)
- **Camelyon16** — total ?; splits: train (-1), test (-1)
- **Camelyon17** — total ?; splits: test (-1)
- **BCNB** — total ?; splits: test (-1)
- **Kvasir-Capsule** — total ?; splits: test (-1)
- **AutoLaparo** — total ?; splits: test (-1)
- **EndoVis18** — total ?; splits: test (-1)
- **EDD 2020** — total ?; splits: test (-1)
- **CT-RATE** — total ?; splits: test (-1)
- **Medical Segmentation Decathlon (MSD)** — total ?; splits: test (-1)
- **CREMI** — total ?; splits: test (-1)
- **AC3/4** — total ?; splits: test (-1)
- **AutoPET-II** — total ?; splits: test (-1)
- **HECKTOR 2022** — total ?; splits: test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve. Measures the model's ability to distinguish between classes across all classification thresholds.
- `Dice score` **(primary)** — range: [0, 1]
  - 2 * |A ∩ B| / (|A| + |B|), where A and B are the predicted and ground truth masks. Measures overlap between binary segmentation predictions and gold labels.
- `IoU (Intersection over Union)` **(primary)** — range: [0, 1]
  - |A ∩ B| / |A ∪ B|. Measures the ratio of the intersection area to the union area between predicted and ground truth masks.
- `Accuracy` — range: [0, 1]
  - Number of correct predictions divided by total number of predictions.
- `F-1 Score` — range: [0, 1]
  - Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall).
- `Jaccard Index` — range: [0, 1]
  - Identical to IoU; measures set similarity between predicted and ground truth masks.
- `VOI (Variation of Information)` — range: [0, ∞)
  - Information-theoretic measure of segmentation error; lower values indicate better agreement between predicted and ground truth segmentations.
- `ARAND (Adjusted Rand)` — range: [0, 1]
  - Adjusted Rand index measuring similarity between two clusterings; lower error rates indicate better segmentation quality.

## Input / output format

**Input**: Medical images (2D X-rays, pathology patches/WSIs, endoscopic frames, 3D CT/PET volumes) resized or processed to fixed input resolutions (e.g., 256x256 or 512x512).

**Output**: Class probabilities or logits for classification tasks; pixel-wise segmentation masks for segmentation tasks.

## Scoring recipe

```python
def compute_metrics(preds, gold):
    # Classification
    auc = roc_auc_score(gold, preds)
    acc = accuracy_score(gold, preds.argmax(axis=1))
    # Segmentation
    intersection = np.sum(pred_mask & gold_mask)
    dice = 2 * intersection / (np.sum(pred_mask) + np.sum(gold_mask))
    iou = intersection / (np.sum(pred_mask | gold_mask))
    return {'AUC': auc, 'Dice': dice, 'IoU': iou}
```

## Common pitfalls

- Performance does not consistently scale with model size or resolution; larger models or higher resolutions can degrade performance.
- Frozen backbone features often lack fine-grained textural details required for histopathology or electron microscopy segmentation.
- 2D slice-wise feature aggregation may be insufficient for volumetric 3D tasks without 3D-specific adapters or pooling strategies.

## Evidence (verbatim from paper)

> As shown in Table [2], DINOv3-L achieves the highest AUC on NIH-14, outperforming the medical-specific BiomedCLIP model. While BiomedCLIP performs best on the RSNA-Pneumonia task, DINOv3 models are close contenders. However, the results also highlight an inconsistent scaling behavior, as seen in Figure [1]. Performance does not reliably improve with larger model sizes or higher input resolutions; for instance, AUC for all models on NIH-14 peaks at a 512x512 resolution before declining.

## Citation

```bibtex
@misc{liu2025dino3medical,
  title={Does DINOv3 Set a New Medical Vision Standard?},
  author={Liu et al. (2025)},
  year={2025},
  note={arXiv:2509.06467}
}
```

- arXiv: 2509.06467