A Survey on Neural Architecture Search
Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati
- Neural Architecture Search
- Automation of Machine Learning
- Deep Learning
- Reinforcement Learning
- Evolutionary Algorithms
- Constrained Optimization
- MultiObjective Optimization
The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architecture search. The choice of the network architecture has proven to be critical, and many advances in deep learning spring from its immediate improvements. However, deep learning techniques are computationally intensive and their application requires a high level of domain knowledge. Therefore, even partial automation of this process helps to make deep learning more accessible to both researchers and practitioners. With this survey, we provide a formalism which unifies and categorizes the landscape of existing methods along with a detailed analysis that compares and contrasts the different approaches. We achieve this via a comprehensive discussion of the commonly adopted architecture search spaces and architecture optimization algorithms based on principles of reinforcement learning and evolutionary algorithms along with approaches that incorporate surrogate and one-shot models. Additionally, we address the new research directions which include constrained and multi-objective architecture search as well as automated data augmentation, optimizer and activation function search.
@article{DBLP:journals/corr/abs-1905-01392, author = {Martin Wistuba and Ambrish Rawat and Tejaswini Pedapati}, title = {A Survey on Neural Architecture Search}, journal = {CoRR}, volume = {abs/1905.01392}, year = {2019}, url = {http://arxiv.org/abs/1905.01392}, archivePrefix = {arXiv}, eprint = {1905.01392}, timestamp = {Mon, 27 May 2019 13:15:00 +0200}, biburl = {https://dblp.org/rec/journals/corr/abs-1905-01392.bib}, bibsource = {dblp computer science bibliography, https://dblp.org} }