An Algorithmic Framework For The Optimization Of Deep Neural Networks Architectures And Hyperparameters

# An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters

**arXiv ID:** 2303.12797
**Authors:** Julie Keisler, El-Ghazali Talbi, Sandra Claudel, Gilles Cabriel
**Published:** 2023-02-27T08:00:33Z
**Abstract:**
In this paper, we propose an algorithmic framework to automatically generate efficient deep neural networks and optimize their associated hyperparameters. The framework is based on evolving directed acyclic graphs (DAGs), defining a more flexible search space than the existing ones in the literature. It allows mixtures of different classical operations: convolutions, recurrences and dense layers, but also more newfangled operations such as self-attention. Based on this search space we propose neighbourhood and evolution search operators to optimize both the architecture and hyper-parameters of our networks. These search operators can be used with any metaheuristic capable of handling mixed search spaces. We tested our algorithmic framework with an evolutionary algorithm on a time series prediction benchmark. The results demonstrate that our framework was able to find models outperforming the established baseline on numerous datasets.

## Skill Description

This skill is generated from the arXiv paper: An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters (2303.12797).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2303.12797](http://arxiv.org/abs/2303.12797v2)