On The Principles Of Relu Networks With One Hidden Layer
ASecurity
**arXiv ID:** 2411.06728 **Authors:** Changcun Huang **Published:** 2024-11-11T05:51:11Z **Abstract:** A neural network with one hidden layer or a two-layer network (regardless of the input layer) is the simplest feedforward neural network, whose mechanism may be the basis of more general network architectures. However, even to this type of simple architecture, it is also a ``black box''; that is, it remains unclear how to interpret the mechanism of its solutions obtained by the back-propagat...
Installs into .claude/skills of the current project.
Sponsors
SSkills DirectorySkills Directory
Know which skills are safe — weekly.
SSkills DirectorySkills Directory
Is that skill safe to install?
Are you the author of On The Principles Of Relu Networks With One Hidden Layer?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-on-the-principles-of-relu-networks-with-one-hidden)
# On the Principles of ReLU Networks with One Hidden Layer
**arXiv ID:** 2411.06728
**Authors:** Changcun Huang
**Published:** 2024-11-11T05:51:11Z
**Abstract:**
A neural network with one hidden layer or a two-layer network (regardless of the input layer) is the simplest feedforward neural network, whose mechanism may be the basis of more general network architectures. However, even to this type of simple architecture, it is also a ``black box''; that is, it remains unclear how to interpret the mechanism of its solutions obtained by the back-propagation algorithm and how to control the training process through a deterministic way. This paper systematically studies the first problem by constructing universal function-approximation solutions. It is shown that, both theoretically and experimentally, the training solution for the one-dimensional input could be completely understood, and that for a higher-dimensional input can also be well interpreted to some extent. Those results pave the way for thoroughly revealing the black box of two-layer ReLU networks and advance the understanding of deep ReLU networks.
## Skill Description
This skill is generated from the arXiv paper: On the Principles of ReLU Networks with One Hidden Layer (2411.06728).
## How to Use
[To be filled in by the user or by future automation]
## References
- [arXiv:2411.06728](http://arxiv.org/abs/2411.06728v2)
