**arXiv ID:** 2010.08251 **Authors:** Andras Horvath, Jalal Al-afandi **Published:** 2020-10-16T08:56:57Z **Abstract:** It is a common assumption that the activation of different layers in neural networks follow Gaussian distribution. This distribution can be transformed using normalization techniques, such as batch-normalization, increasing convergence speed and improving accuracy. In this paper we would like to demonstrate, that activations do not necessarily follow Gaussian distribution in...
Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of Filtered Batch Normalization?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-filtered-batch-normalization)
# Filtered Batch Normalization
**arXiv ID:** 2010.08251
**Authors:** Andras Horvath, Jalal Al-afandi
**Published:** 2020-10-16T08:56:57Z
**Abstract:**
It is a common assumption that the activation of different layers in neural networks follow Gaussian distribution. This distribution can be transformed using normalization techniques, such as batch-normalization, increasing convergence speed and improving accuracy. In this paper we would like to demonstrate, that activations do not necessarily follow Gaussian distribution in all layers. Neurons in deeper layers are more selective and specific which can result extremely large, out-of-distribution activations.
We will demonstrate that one can create more consistent mean and variance values for batch normalization during training by filtering out these activations which can further improve convergence speed and yield higher validation accuracy.
## Skill Description
This skill is generated from the arXiv paper: Filtered Batch Normalization (2010.08251).
## How to Use
[To be filled in by the user or by future automation]
## References
- [arXiv:2010.08251](http://arxiv.org/abs/2010.08251v1)