Myriadal Eval

---
name: myriadal-eval
description: Evaluates active few-shot learning frameworks for histopathology image classification under extremely tight annotation budgets (1, 5, and 10 labeled samples). It probes how well uncertainty-based diversity sampling and self-supervised contrastive pretraining can reduce sample redundancy and improve classification performance compared to standard few-shot and active learning baselines. Use when the user wants to benchmark on NCT-CRC-HE-100K, BreaKHis, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2310.16161
  bibtex_key: schiavone2023myriadal
  confidence: high
---

# myriadal-eval

> MyriadAL: Active Few Shot Learning for Histopathology — Schiavone et al. (2023) (arXiv:2310.16161, 2023)

## What this evaluates

Evaluates active few-shot learning frameworks for histopathology image classification under extremely tight annotation budgets (1, 5, and 10 labeled samples). It probes how well uncertainty-based diversity sampling and self-supervised contrastive pretraining can reduce sample redundancy and improve classification performance compared to standard few-shot and active learning baselines.

## Datasets

- **NCT-CRC-HE-100K** — total 100000; splits: train (-1), test (-1)
- **BreaKHis** — total 9109; splits: train (-1), test (-1)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Proportion of correctly classified test images out of the total test set size.
- `Macro F1 Score` — range: percent
  - Unweighted mean of the F1 scores computed independently for each class, where F1 is the harmonic mean of precision and recall per class.

## Input / output format

**Input**: RGB histopathology image patches (224×224 for NCT, 700×460 for BreaKHis) with ground-truth class labels (9 classes for NCT, 8 classes for BreaKHis).

**Output**: Predicted class label for each test image.

## Scoring recipe

```python
correct = sum(p == g for p, g in zip(predictions, gold))
accuracy = correct / len(gold)
f1_scores = []
for cls in classes:
    tp = sum(p == cls and g == cls for p, g in zip(predictions, gold))
    fp = sum(p == cls and g != cls for p, g in zip(predictions, gold))
    fn = sum(p != cls and g == cls for p, g in zip(predictions, gold))
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    f1_scores.append(f1)
macro_f1 = sum(f1_scores) / len(classes)
```

## Common pitfalls

- Active learning baselines require an initial seed of K randomly chosen labeled samples; failing to fix this seed across runs breaks comparability.
- All results are reported as mean ± standard deviation over 3 random seeds; single-run evaluations will not match the paper's numbers.
- The 80:20 train/test split is performed randomly without explicit stratification, which can cause severe class imbalance in 1-shot or 5-shot settings.

## Evidence (verbatim from paper)

> Here, we use classification accuracy and macro F1 scores as the performance metrics.

## Citation

```bibtex
@misc{schiavone2023myriadal,
  title={MyriadAL: Active Few Shot Learning for Histopathology},
  author={Schiavone et al. (2023)},
  year={2023},
  note={arXiv:2310.16161}
}
```

- arXiv: 2310.16161