Architect Regularize And Replay Arr A Flexible Hybrid Approach For Continual Learning

# Architect, Regularize and Replay (ARR): a Flexible Hybrid Approach for Continual Learning

**arXiv ID:** 2301.02464
**Authors:** Vincenzo Lomonaco, Lorenzo Pellegrini, Gabriele Graffieti, Davide Maltoni
**Published:** 2023-01-06T11:22:59Z
**Abstract:**
In recent years we have witnessed a renewed interest in machine learning methodologies, especially for deep representation learning, that could overcome basic i.i.d. assumptions and tackle non-stationary environments subject to various distributional shifts or sample selection biases. Within this context, several computational approaches based on architectural priors, regularizers and replay policies have been proposed with different degrees of success depending on the specific scenario in which they were developed and assessed. However, designing comprehensive hybrid solutions that can flexibly and generally be applied with tunable efficiency-effectiveness trade-offs still seems a distant goal. In this paper, we propose "Architect, Regularize and Replay" (ARR), an hybrid generalization of the renowned AR1 algorithm and its variants, that can achieve state-of-the-art results in classic scenarios (e.g. class-incremental learning) but also generalize to arbitrary data streams generated from real-world datasets such as CIFAR-100, CORe50 and ImageNet-1000.

## Skill Description

This skill is generated from the arXiv paper: Architect, Regularize and Replay (ARR): a Flexible Hybrid Approach for Continual Learning (2301.02464).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2301.02464](http://arxiv.org/abs/2301.02464v1)