Fsl Episodic Eval

---
name: fsl-episodic-eval
description: This benchmark evaluates few-shot generalization capability by measuring classification accuracy in an episodic setting where models must recognize novel classes using only a few labeled support examples. It probes the model's ability to adapt quickly to new categories and its robustness to test-time data augmentation. Use when the user wants to benchmark on miniImagenet, tieredImagenet, CUB, Animals, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.00114
  bibtex_key: bai20261sdaug
  confidence: high
---

# fsl-episodic-eval

> 1S-DAug: One-Shot Data Augmentation for Robust Few-Shot Generalization — Bai et al. (2026) (arXiv:2602.00114, 2026)

## What this evaluates

This benchmark evaluates few-shot generalization capability by measuring classification accuracy in an episodic setting where models must recognize novel classes using only a few labeled support examples. It probes the model's ability to adapt quickly to new categories and its robustness to test-time data augmentation.

## Datasets

- **miniImagenet** — total ?; splits: train (-1), val (-1), test (-1)
- **tieredImagenet** — total ?; splits: train (-1), val (-1), test (-1)
- **CUB** — total ?; splits: train (-1), val (-1), test (-1)
- **Animals** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly classified query images across sampled episodes, computed as (correct predictions / total query images) × 100. Reported as mean accuracy with 95% confidence intervals.

## Input / output format

**Input**: Episodic few-shot setup: a support set containing K images per class (K=1 or 5) and a query set containing multiple images per class. Each instance consists of image pairs/groups passed through a backbone encoder to extract embeddings.

**Output**: Class label predictions for each query image, determined by nearest-neighbor matching (Euclidean distance or cosine similarity) to class prototypes in the embedding space.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    total = len(gold_labels)
    return (correct / total) * 100
```

## Common pitfalls

- Augmenting only the support set or only the query set causes distribution mismatch and degrades performance; both must be augmented identically.
- Removing image conditioning during diffusion generation or using full noise (η=1.0) severely compromises class faithfulness and accuracy.
- Traditional geometric test-time augmentations (rotations, affine warps) yield significantly lower accuracy (~57.9%) compared to diffusion-based methods.

## Evidence (verbatim from paper)

> We sample 15,000 5-way-1/5-shot queries and report mean accuracy with 95% confidence intervals across episodes. This follows the standard protocol used in related FSL work (Ye et al., [2020]; Snell et al., [2017]).

## Citation

```bibtex
@misc{bai20261sdaug,
  title={1S-DAug: One-Shot Data Augmentation for Robust Few-Shot Generalization},
  author={Bai et al. (2026)},
  year={2026},
  note={arXiv:2602.00114}
}
```

- arXiv: 2602.00114