Efficient Offpolicy Reinforcement Learning Via Braininspired Computing

# Efficient Off-Policy Reinforcement Learning via Brain-Inspired Computing

**arXiv ID:** 2205.06978
**Authors:** Yang Ni, Danny Abraham, Mariam Issa, Yeseong Kim, Pietro Mercati, Mohsen Imani
**Published:** 2022-05-14T05:50:54Z
**Abstract:**
Reinforcement Learning (RL) has opened up new opportunities to enhance existing smart systems that generally include a complex decision-making process. However, modern RL algorithms, e.g., Deep Q-Networks (DQN), are based on deep neural networks, resulting in high computational costs. In this paper, we propose QHD, an off-policy value-based Hyperdimensional Reinforcement Learning, that mimics brain properties toward robust and real-time learning. QHD relies on a lightweight brain-inspired model to learn an optimal policy in an unknown environment. On both desktop and power-limited embedded platforms, QHD achieves significantly better overall efficiency than DQN while providing higher or comparable rewards. QHD is also suitable for highly-efficient reinforcement learning with great potential for online and real-time learning. Our solution supports a small experience replay batch size that provides 12.3 times speedup compared to DQN while ensuring minimal quality loss. Our evaluation shows QHD capability for real-time learning, providing 34.6 times speedup and significantly better quality of learning than DQN.

## Skill Description

This skill is generated from the arXiv paper: Efficient Off-Policy Reinforcement Learning via Brain-Inspired Computing (2205.06978).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2205.06978](http://arxiv.org/abs/2205.06978v3)