Can A Hebbianlike Learning Rule Be Avoiding The Curse Of Dimensionality In Sparse Distributed Data

# Can a Hebbian-like learning rule be avoiding the curse of dimensionality in sparse distributed data?

**arXiv ID:** 2208.12564
**Authors:** Maria Osório, Luís Sa-Couto, Andreas Wichert
**Published:** 2022-07-20T17:08:10Z
**Abstract:**
It is generally assumed that the brain uses something akin to sparse distributed representations. These representations, however, are high-dimensional and consequently they affect classification performance of traditional Machine Learning models due to "the curse of dimensionality". In tasks for which there is a vast amount of labeled data, Deep Networks seem to solve this issue with many layers and a non-Hebbian backpropagation algorithm. The brain, however, seems to be able to solve the problem with few layers. In this work, we hypothesize that this happens by using Hebbian learning. Actually, the Hebbian-like learning rule of Restricted Boltzmann Machines learns the input patterns asymmetrically. It exclusively learns the correlation between non-zero values and ignores the zeros, which represent the vast majority of the input dimensionality. By ignoring the zeros "the curse of dimensionality" problem can be avoided. To test our hypothesis, we generated several sparse datasets and compared the performance of a Restricted Boltzmann Machine classifier with some Backprop-trained networks. The experiments using these codes confirm our initial intuition as the Restricted Boltzmann Machine shows a good generalization performance, while the Neural Networks trained with the backpropagation algorithm overfit the training data.

## Skill Description

This skill is generated from the arXiv paper: Can a Hebbian-like learning rule be avoiding the curse of dimensionality in sparse distributed data? (2208.12564).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2208.12564](http://arxiv.org/abs/2208.12564v1)