Implicit Regularization With Polynomial Growth In Deep Tensor Factorization

# Implicit Regularization with Polynomial Growth in Deep Tensor Factorization

**arXiv ID:** 2207.08942
**Authors:** Kais Hariz, Hachem Kadri, Stéphane Ayache, Maher Moakher, Thierry Artières
**Published:** 2022-07-18T21:04:37Z
**Abstract:**
We study the implicit regularization effects of deep learning in tensor factorization. While implicit regularization in deep matrix and 'shallow' tensor factorization via linear and certain type of non-linear neural networks promotes low-rank solutions with at most quadratic growth, we show that its effect in deep tensor factorization grows polynomially with the depth of the network. This provides a remarkably faithful description of the observed experimental behaviour. Using numerical experiments, we demonstrate the benefits of this implicit regularization in yielding a more accurate estimation and better convergence properties.

## Skill Description

This skill is generated from the arXiv paper: Implicit Regularization with Polynomial Growth in Deep Tensor Factorization (2207.08942).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2207.08942](http://arxiv.org/abs/2207.08942v2)