Fscil Eval

---
name: fscil-eval
description: Probes a model's ability to learn new classes incrementally in a few-shot setting while retaining knowledge of previously learned classes, measuring resistance to catastrophic forgetting. Use when the user wants to benchmark on miniImageNet, CIFAR-100, CUB-200, or asks about evaluating this task. Reports Average accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2302.03004
  bibtex_key: yang2023neural
  confidence: high
---

# fscil-eval

> Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning — Yang et al. (2023) (arXiv:2302.03004, 2023)

## What this evaluates

Probes a model's ability to learn new classes incrementally in a few-shot setting while retaining knowledge of previously learned classes, measuring resistance to catastrophic forgetting.

## Datasets

- **miniImageNet** — total ?; splits: train (-1), val (-1), test (-1)
- **CIFAR-100** — total ?; splits: train (-1), test (-1)
- **CUB-200** — total ?; splits: train (-1), test (-1)

## Metrics

- `Average accuracy` **(primary)** — range: percent
  - The arithmetic mean of the classification accuracy across all incremental sessions (base + K new sessions).
- `Final session accuracy` — range: percent
  - The classification accuracy evaluated on the test set of the last incremental session.
- `Performance drop` — range: percent
  - The difference in accuracy between the first session and the last session (First session Acc. - Last session Acc.), measuring catastrophic forgetting.

## Input / output format

**Input**: Image samples with class labels, presented sequentially in incremental sessions. Each session provides a few-shot support set and a query set for evaluation.

**Output**: Predicted class label for each query image.

## Scoring recipe

```python
def compute_metrics(session_accuracies):
    avg_acc = sum(session_accuracies) / len(session_accuracies)
    final_acc = session_accuracies[-1]
    perf_drop = session_accuracies[0] - session_accuracies[-1]
    return {'Average accuracy': avg_acc, 'Final session accuracy': final_acc, 'Performance drop': perf_drop}
```

## Common pitfalls

- FSCIL evaluation requires testing on *all* classes seen so far in each session, not just the newly introduced ones.
- Average accuracy must be computed across all sessions, not just the final one, to properly measure forgetting.
- Few-shot support set size (e.g., 1-shot vs 5-shot) must be strictly controlled, as it heavily impacts accuracy.

## Evidence (verbatim from paper)

> “Average Acc.” is the average accuracy of all sessions. “Final Improv.” calculates the improvement of our method in the last session.

## Citation

```bibtex
@misc{yang2023neural,
  title={Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning},
  author={Yang et al. (2023)},
  year={2023},
  note={arXiv:2302.03004}
}
```

- arXiv: 2302.03004