Neural Architecture Search with Reinforcement Learning · HackerPapers discussions

Neural Architecture Search with Reinforcement Learning — mentioned by kripke

Fri, 13 Jul 2018 18:55:47 GMT

Not GP, but you are probably looking for the "Neural Architecture Search" series [1] [2] [3]. First one uses something like 1k GPUs for a month, next one is a bit more reasonable, and the last one actually has a reasonable training time, and has comparable performance to DARTS (see ENAS in comparison tables). 1: https://arxiv.org/abs/1611.01578 2: https://arxiv.org/abs/1707.07012 3: https://arxiv.org/abs/1802.03268

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by ashelmire

Fri, 13 Jul 2018 18:45:24 GMT

Cool, was thinking about this subject earlier today. Abstract: >This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent. Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering…

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by RoboTeddy

Mon, 04 Jun 2018 20:35:53 GMT

This question isn't silly! People have actually trained neural networks to generate other neural networks. The "designer" network is trained using reinforcement learning, with a reward signal that depends on the efficacy of the networks it creates. See: Neural Architecture Search with Reinforcement Learning (https://arxiv.org/abs/1611.01578) It looks like they trained the "designer" network by trainning 15,000 deep learning models (800 at a time in parallel over 800 GPUs)

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by michaelgreen

Wed, 14 Mar 2018 22:50:21 GMT

Jeff Dean talks about AutoML using RL and in the paper " Neural Architecture Search with Reinforcement Learning" it also talks about this. Also it seems different from more traditional hyperparameter optimization because it makes novel cells. So the structure of the network isn't limited to our existing library of layers/cells. https://arxiv.org/abs/1611.01578 https://youtu.be/HcStlHGpjN8?t=2073

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by ollin

Mon, 06 Nov 2017 14:34:56 GMT

Yeah, I'm confused that this is the top comment; it's factually incorrect. NASNet is an example of a result of AutoML. To quote the Google blogpost on NASNet: >In Learning Transferable Architectures for Scalable Image Recognition, we apply AutoML to the ImageNet image classification and COCO object detection dataset... AutoML was able to find the best layers that work well on CIFAR-10 but work well on ImageNet classification and COCO object detection. These two layers are combined to form a novel architecture, which we called “NASNet”.…

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by chriskanan

Fri, 14 Jul 2017 18:35:50 GMT

There has been some work related to what you are describing: Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474 Related (but less so), there are also some papers about learning neural network architectures: Designing Neural Network Architectures using Reinforcement Learning https://arxiv.org/abs/1611.02167 Neural Architecture Search with Reinforcement Learning https://arxiv.org/abs/1611.01578

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by ganfortran

Fri, 31 Mar 2017 19:02:39 GMT

Neural Architecture Search with Reinforcement Learning https://arxiv.org/abs/1611.01578 Too bad, it is already there :P

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by sarthakjain

Sun, 05 Feb 2017 11:54:11 GMT

Deep learning architectures built by machines (so we no longer have to design architecture to solve problems) https://arxiv.org/abs/1611.01578 Transfer Learning (so we need less data to build models) http://ftp.cs.wisc.edu/machine-learning/shavlik-group/torrey... Generative adversarial networks (so computers can get human like abilities at generating content) https://papers.nips.cc/paper/5423-generative-adversarial-net...

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — HN submission

Thu, 19 Jan 2017 20:38:25 GMT

Neural Architecture Search with Reinforcement Learning

116 points; 38 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by lucidrains

Tue, 03 Jan 2017 00:55:52 GMT

Other equally exciting papers that relates to learning to learn in DL. "Neural Architecture Search with Reinforcement Learning" https://arxiv.org/abs/1611.01578 "RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning" https://arxiv.org/abs/1611.02779 "Designing Neural Network Architectures using Reinforcement Learning" https://arxiv.org/abs/1611.02167

Paper and discussion history. A mention is not an endorsement.

Neural Architecture Search with Reinforcement Learning — mentioned by nomailing

Sun, 04 Dec 2016 01:47:31 GMT

Interesting. The concept seems to be very similar to this one: https://arxiv.org/abs/1611.01578

Paper and discussion history. A mention is not an endorsement.