Training Neural Networks With Optimal Doublebayesian Learning

# Training Neural Networks with Optimal Double-Bayesian Learning

**arXiv ID:** 2605.20009
**Authors:** Vy Bui, Hang Yu, Karthik Kantipudi, Ziv Yaniv, Stefan Jaeger
**Published:** 2026-05-19T15:39:36Z
**Abstract:**
Backpropagation with gradient descent is a common optimization strategy employed by most neural network architectures in machine learning. However, finding optimal hyperparameters to guide training has proven challenging. While it is widely acknowledged that selecting appropriate parameters is crucial for avoiding overfitting and achieving unbiased outcomes, this choice remains largely based on empirical experiments and experience. This paper presents a new probabilistic framework for the learning rate, a key parameter in stochastic gradient descent. The framework develops classic Bayesian statistics into a double-Bayesian decision mechanism involving two antagonistic Bayesian processes. A theoretically optimal learning rate can be derived from these two processes and used for stochastic gradient descent. Experiments across various classification, segmentation, and detection tasks corroborate the practical significance of the theoretically derived learning rate. The paper also discusses the ramifications of the proposed double-Bayesian framework for network training and model performance.

## Skill Description

This skill is generated from the arXiv paper: Training Neural Networks with Optimal Double-Bayesian Learning (2605.20009).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2605.20009](http://arxiv.org/abs/2605.20009v1)