A Significantly Better Class Of Activation Functions Than Relu Like Activation Functions
ASecurity**arXiv ID:** 2405.04459 **Authors:** Mathew Mithra Noel, Yug Oswal **Published:** 2024-05-07T16:24:03Z **Abstract:** This paper introduces a significantly better class of activation functions than the almost universally used ReLU like and Sigmoidal class of activation functions. Two new activation functions referred to as the Cone and Parabolic-Cone that differ drastically from popular activation functions and significantly outperform these on the CIFAR-10 and Imagenette benchmmarks are prop...
- 3 stars
- 0 votes
- 0 copies
- 4 views
- Added September 11, 2026
Security analysis
100/100npx -y skills add hiyenwong/ai_collection --skill a-significantly-better-class-of-activation-functions-than-relu-like-activation-functions --agent claude-codeAre you the author of A Significantly Better Class Of Activation Functions Than Relu Like Activation Functions?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-a-significantly-better-class-of-activation-functio)# A Significantly Better Class of Activation Functions Than ReLU Like Activation Functions
**arXiv ID:** 2405.04459
**Authors:** Mathew Mithra Noel, Yug Oswal
**Published:** 2024-05-07T16:24:03Z
**Abstract:**
This paper introduces a significantly better class of activation functions than the almost universally used ReLU like and Sigmoidal class of activation functions. Two new activation functions referred to as the Cone and Parabolic-Cone that differ drastically from popular activation functions and significantly outperform these on the CIFAR-10 and Imagenette benchmmarks are proposed. The cone activation functions are positive only on a finite interval and are strictly negative except at the end-points of the interval, where they become zero. Thus the set of inputs that produce a positive output for a neuron with cone activation functions is a hyperstrip and not a half-space as is the usual case. Since a hyper strip is the region between two parallel hyper-planes, it allows neurons to more finely divide the input feature space into positive and negative classes than with infinitely wide half-spaces. In particular the XOR function can be learn by a single neuron with cone-like activation functions. Both the cone and parabolic-cone activation functions are shown to achieve higher accuracies with significantly fewer neurons on benchmarks. The results presented in this paper indicate that many nonlinear real-world datasets may be separated with fewer hyperstrips than half-spaces. The Cone and Parabolic-Cone activation functions have larger derivatives than ReLU and are shown to significantly speedup training.
## Skill Description
This skill is generated from the arXiv paper: A Significantly Better Class of Activation Functions Than ReLU Like Activation Functions (2405.04459).
## How to Use
[To be filled in by the user or by future automation]
## References
- [arXiv:2405.04459](http://arxiv.org/abs/2405.04459v1)
Attribution
Related skills
View all in tools →Comments
Loading comments…
Newsletter
Get the weekly list of safe new skills, and the ones we pulled.