Architecture engineering in deep neural networks (DNNs), or more in general in artificial neural networks (ANNs), often requires significant effort. Indeed, given a specific classification (or regression) problem the first point to address concerning the definition of the network's architecture (i.e., the number of hidden layers, the number of neurons and the activation functions for each layer). Typically, to identify the best architecture a large number of different configuration are trained and then compared.
Moreover, for each architecture, it is necessary to tune the weights and bias, as well as the hyperparameters of the model (e.g., the learning rate, the regularization coefficient, etc.). As a result, choosing the best architecture is a time-consuming process often based on past experience on a similar problem. In this work, we propose a methodology to dynamically adapt a neural network architecture. The aim of this study is twofold: on the one hand, show that it is possible to modify during the training the architecture exploiting its history. On the other hand, dynamically determine the best architecture for a given classification (or regression) problem.
