Learning to learn by gradient descent by gradient descent

How does it compare to previous work on learning to learn? I don't see it referenced https://arxiv.org/abs/1606.04474

jn2clarkMar 26, 2025View on HNParent context

> So, in case of (stochastic) gradient descent to train a neural network, that would be the neural network that you want to use AD on, not the gradient descent algorithm. Obviously. The specific proposal of differential programming as a paradigm that goes beyond simple…

radarsat1Aug 1, 2022View on HNParent context

Can’t pass on the opportunity to share one of my favorite (and topical!) paper titles: https://arxiv.org/abs/1606.04474

samatmanOct 3, 2019View on HNParent context

Your algorithm description sounds like a variant of greedy local search with a NN heuristic. While I personaly do not know of a paper that does this kind of local search (I have not looked), I do want to add that there is a rich literature on "learning to learning", including…

human_scientistMar 23, 2019View on HNParent context

There has been some work related to what you are describing: Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474 Related (but less so), there are also some papers about learning neural network architectures: Designing Neural Network…

chriskananJul 14, 2017View on HNParent context

Besides Googling for tricks, another way of learning LaTeX is to download the source code for nice-looking papers that you find on the arXiv. For example, for "Learning to learn by gradient descent by gradient descent" arXiv:1606.04474 (https://arxiv.org/abs/1606.04474), if you…

greeneggsApr 30, 2017View on HNParent context

I think it's important to note that all of these methods have been largely superseded by deep learning techniques. For example, we can now directly learn algorithms such as gradient descent[1] and classical inverse problems like superresolution are solvable with deep networks…

digitalemble72Feb 1, 2017View on HNParent context

Learning to learn by gradient descent by gradient descent: https://arxiv.org/abs/1606.04474

aisoftengJan 20, 2017View on HNParent context

Disclaimer: Interested in NN, but no real knowledge yet. A ML-learning group at my university recently discussed Learning to learn by gradient descent by gradient descent [1], the topic seems related. This indicates to me that at least some progress was made. [1]…

LeanderKJan 10, 2017View on HNParent context

The obvious citation is the wonderfully titled Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474

joshvmJan 11, 2017View on HNParent context

GANs (and InfoGAN[0] in particular) has shown there are still pretty huge breakthoughs out there. Some other things you may be interested in: Learning to learn by gradient descent by gradient descent[1] Neural Architecture Search with Reinforcement Learning[2] [0]…

nlJan 11, 2017View on HNParent context

"Learning to learn by gradient descent by gradient descent" https://arxiv.org/abs/1606.04474

panarkyJan 3, 2017View on HNParent context

I assume this https://arxiv.org/abs/1606.04474 (since the title is basically the quoted text)

detaroDec 22, 2016View on HNParent context

Blog spam pass through to the original: http://sebastianruder.com/optimizing-gradient-descent/ Useful reference, 6 months old. Side note, anybody aware of any implementations of the "learning to learn gradient descent by gradient descent" work [0]? I'd love to boost my training…

highdNov 9, 2016View on HNParent context

> Humans can design good features for the ML algorithm to use though For now. Articles like [1] make me wonder whether this current generation of programmers might be the last. [1] https://arxiv.org/abs/1606.04474

quotemstrAug 6, 2016View on HNParent context