Design and evaluation of neural architecture, using reinforcement learning and distributed computing

Abstract

Relying on neural networks, Deep Learning has solved many difficult machine learning problems. Although the optimization of the network’s weights is of paramount importance for its performance, the network’s architecture has been shown to contribute significantly as well. While the former is a relatively straightforward, numerical method, the latter remains a procedure heavily relying on human expertise and experimentation. In this study, we try to reproduce research conducted on Neural Architecture Search by utilizing Reinforcement Learning techniques as well as try different approaches. We implement a small-scale framework, using Double Deep Q-learning Networks while applying it to the well-known MNIST dataset of hand-written digits. We then apply Synchronous Advantage Actor-Critic for both discrete as well as continuous action spaces. Finally, we experiment with partial training of the neural networks, in order to reduce the computational resources required to evaluate the architectures. The aim of the study is to verify the viability of using distributed Reinforcement Learning in Neural Architecture Search, as well as the feasibility of using partial training in order to evaluate a neural network’s architecture. We find that distributed Reinforcement Learning can indeed be used to find optimal architectures as well as the use of partial training in order to evaluate an architecture.

The relevant python code is hosted in https://github.com/GeorgeKyriakides/NAS