Po Meta Dataset Eval

---
name: po-meta-dataset-eval
description: Evaluates few-shot representation learning under partial observability. Models must match query image views to their underlying source images using only partial support views (≤50% coverage) and viewpoint coordinates. Use when the user wants to benchmark on PO-Meta-Dataset, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2301.13136
  bibtex_key: jelley2023poem
  confidence: high
---

# po-meta-dataset-eval

> Contrastive Meta-Learning for Partially Observable Few-Shot Learning — Jelley et al. (2023) (arXiv:2301.13136, 2023)

## What this evaluates

Evaluates few-shot representation learning under partial observability. Models must match query image views to their underlying source images using only partial support views (≤50% coverage) and viewpoint coordinates.

## Datasets

- **PO-Meta-Dataset** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/AdamJelley/POEM

## Metrics

- `accuracy` **(primary)** — range: percent
  - Standard few-shot classification accuracy: the proportion of query views correctly matched to their source sample ID from the support set.

## Input / output format

**Input**: Partial views (random crops) of an image, with viewpoint coordinates. Support set contains up to 50% of the image split into multiple views; query set contains additional views.

**Output**: Predicted sample ID (class) corresponding to the query view.

## Scoring recipe

```python
correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if pred == gold:
        correct += 1
    total += 1
return correct / total
```

## Common pitfalls

- Support views cover at most 50% of the image, not the full image.
- Viewpoint coordinates are provided and should be used to locate views.
- Uses Meta-Dataset's specific sampling procedure (uniform class sampling, balanced query, unbalanced support).

## Evidence (verbatim from paper)

> Few-shot classification accuracies on our adapted Meta-Dataset benchmark, PO-Meta-Dataset. All learners use a ResNet-18 model pre-trained on ImageNet, with MLP heads to incorporate view information.

## Citation

```bibtex
@misc{jelley2023poem,
  title={Contrastive Meta-Learning for Partially Observable Few-Shot Learning},
  author={Jelley et al. (2023)},
  year={2023},
  note={arXiv:2301.13136}
}
```

- arXiv: 2301.13136