Qensemble For Offline Rl Dont Scale The Ensemble Scale The Batch Size
ASecurity
**arXiv ID:** 2211.11092 **Authors:** Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov, Dmitry Akimov, Sergey Kolesnikov **Published:** 2022-11-20T21:48:25Z **Abstract:** Training large neural networks is known to be time-consuming, with the learning duration taking days or even weeks. To address this problem, large-batch optimization was introduced. This approach demonstrated that scaling mini-batch sizes with appropriate learning rate adjustments can speed up the training process by ord...
Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of Qensemble For Offline Rl Dont Scale The Ensemble Scale The Batch Size?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-qensemble-for-offline-rl-dont-scale-the-ensemble-s)
# Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size
**arXiv ID:** 2211.11092
**Authors:** Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov, Dmitry Akimov, Sergey Kolesnikov
**Published:** 2022-11-20T21:48:25Z
**Abstract:**
Training large neural networks is known to be time-consuming, with the learning duration taking days or even weeks. To address this problem, large-batch optimization was introduced. This approach demonstrated that scaling mini-batch sizes with appropriate learning rate adjustments can speed up the training process by orders of magnitude. While long training time was not typically a major issue for model-free deep offline RL algorithms, recently introduced Q-ensemble methods achieving state-of-the-art performance made this issue more relevant, notably extending the training duration. In this work, we demonstrate how this class of methods can benefit from large-batch optimization, which is commonly overlooked by the deep offline RL community. We show that scaling the mini-batch size and naively adjusting the learning rate allows for (1) a reduced size of the Q-ensemble, (2) stronger penalization of out-of-distribution actions, and (3) improved convergence time, effectively shortening training duration by 3-4x times on average.
## Skill Description
This skill is generated from the arXiv paper: Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size (2211.11092).
## How to Use
[To be filled in by the user or by future automation]
## References
- [arXiv:2211.11092](http://arxiv.org/abs/2211.11092v2)