Few Shot Video Classification Eval

---
name: few-shot-video-classification-eval
description: Evaluates a model's ability to classify video actions from a small number of labeled examples per class. It probes few-shot learning capabilities by measuring accuracy across thousands of randomly sampled episodes on standard video benchmarks. Use when the user wants to benchmark on Kinetics, Something-Something V2, complete-Kinetics, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2110.12358
  bibtex_key: zhu2021closer
  confidence: high
---

# few-shot-video-classification-eval

> A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark — Zhu et al. (2021) (arXiv:2110.12358, 2021)

## What this evaluates

Evaluates a model's ability to classify video actions from a small number of labeled examples per class. It probes few-shot learning capabilities by measuring accuracy across thousands of randomly sampled episodes on standard video benchmarks.

## Datasets

- **Kinetics** — total 10000; splits: train (6400), val (1200), test (2400)
- **Something-Something V2** — total 71796; splits: train (67013), val (1926), test (2857)
- **complete-Kinetics** — total 52925; splits: train (49325), val (1200), test (2400)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Mean classification accuracy computed over 10,000 randomly sampled test episodes. Calculated as the number of correctly classified query videos divided by the total number of episodes.

## Input / output format

**Input**: Support set: n classes (default 5) with k labeled video instances each. Query set: 1 video instance. Videos are preprocessed to 224x224 resolution with T=8 frames sampled.

**Output**: Predicted class label for the query video.

## Scoring recipe

```python
correct = 0
for _ in range(10000):
    support, query = sample_episode(n_classes=5, k_shot=k)
    pred = model.predict(query, support)
    if pred == query.label:
        correct += 1
return (correct / 10000) * 100
```

## Common pitfalls

- Using ImageNet pre-trained weights introduces label leakage due to semantic overlap with novel action classes, violating the few-shot assumption.
- Training on the standard small Kinetics split without pre-training leads to severe overfitting; the authors recommend the complete-Kinetics benchmark for fair evaluation.
- Meta-learning methods often prioritize temporal alignment over feature representation learning, which can be outperformed by simple linear classifiers on deeper backbones.

## Evidence (verbatim from paper)

> In each episode, we randomly sample  $n$  classes with each class containing  $k$  labeled instances as our support set and 1 instance as our query set. In the testing stage, we report the mean accuracy by randomly sampling 10,000 episodes in the experiments.

## Citation

```bibtex
@misc{zhu2021closer,
  title={A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark},
  author={Zhu et al. (2021)},
  year={2021},
  note={arXiv:2110.12358}
}
```

- arXiv: 2110.12358