Learning to learn by gradient descent by gradient descent · HackerPapers discussions

Learning to learn by gradient descent by gradient descent — mentioned by jn2clark

Wed, 26 Mar 2025 03:28:16 GMT

How does it compare to previous work on learning to learn? I don't see it referenced https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by radarsat1

Mon, 01 Aug 2022 05:26:41 GMT

> So, in case of (stochastic) gradient descent to train a neural network, that would be the neural network that you want to use AD on, not the gradient descent algorithm. Obviously. The specific proposal of differential programming as a paradigm that goes beyond simple applications of gradient descent for optimising NNs is exactly to apply gradient descent to optimise learning (and other) algorithms themselves. This may be to select hyperparameters, or to select among a family of algorithms, or to include differentiable constraints in the solver, etc. I wonder, are you familiar with now…

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by samatman

Thu, 03 Oct 2019 00:45:59 GMT

Can’t pass on the opportunity to share one of my favorite (and topical!) paper titles: https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by human_scientist

Sat, 23 Mar 2019 13:38:24 GMT

Your algorithm description sounds like a variant of greedy local search with a NN heuristic. While I personaly do not know of a paper that does this kind of local search (I have not looked), I do want to add that there is a rich literature on "learning to learning", including "learning to optimize" approaches, e.g., https://arxiv.org/abs/1606.04474.

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by chriskanan

Fri, 14 Jul 2017 18:35:50 GMT

There has been some work related to what you are describing: Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474 Related (but less so), there are also some papers about learning neural network architectures: Designing Neural Network Architectures using Reinforcement Learning https://arxiv.org/abs/1611.02167 Neural Architecture Search with Reinforcement Learning https://arxiv.org/abs/1611.01578

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by greeneggs

Sun, 30 Apr 2017 20:27:51 GMT

Besides Googling for tricks, another way of learning LaTeX is to download the source code for nice-looking papers that you find on the arXiv. For example, for "Learning to learn by gradient descent by gradient descent" arXiv:1606.04474 (https://arxiv.org/abs/1606.04474), if you click on Download->Other formats, you get the latex source code and in this case also the Omnigraffle files for the diagrams.

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by digitalemble72

Wed, 01 Feb 2017 03:30:05 GMT

I think it's important to note that all of these methods have been largely superseded by deep learning techniques. For example, we can now directly learn algorithms such as gradient descent[1] and classical inverse problems like superresolution are solvable with deep networks [2]. While there still may be a role for tools like CVX, I anticipate all future progress will come from end-to-end differentiable systems. [1] https://arxiv.org/abs/1606.04474 [2] https://arxiv.org/abs/1501.00092

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by aisofteng

Fri, 20 Jan 2017 01:42:26 GMT

Learning to learn by gradient descent by gradient descent: https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by nl

Wed, 11 Jan 2017 03:07:04 GMT

GANs (and InfoGAN[0] in particular) has shown there are still pretty huge breakthoughs out there. Some other things you may be interested in: Learning to learn by gradient descent by gradient descent[1] Neural Architecture Search with Reinforcement Learning[2] [0] https://arxiv.org/abs/1606.03657, https://arxiv.org/pdf/1701.00160.pdf [1] https://arxiv.org/abs/1606.04474 [2] https://openreview.net/pdf?id=r1Ue8Hcxg

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by joshvm

Wed, 11 Jan 2017 03:06:42 GMT

The obvious citation is the wonderfully titled Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by LeanderK

Tue, 10 Jan 2017 20:40:26 GMT

Disclaimer: Interested in NN, but no real knowledge yet. A ML-learning group at my university recently discussed Learning to learn by gradient descent by gradient descent [1], the topic seems related. This indicates to me that at least some progress was made. [1] https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by panarky

Tue, 03 Jan 2017 00:40:02 GMT

"Learning to learn by gradient descent by gradient descent" https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by detaro

Thu, 22 Dec 2016 11:44:35 GMT

I assume this https://arxiv.org/abs/1606.04474 (since the title is basically the quoted text)

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by highd

Wed, 09 Nov 2016 21:08:38 GMT

Blog spam pass through to the original: http://sebastianruder.com/optimizing-gradient-descent/ Useful reference, 6 months old. Side note, anybody aware of any implementations of the "learning to learn gradient descent by gradient descent" work [0]? I'd love to boost my training time, but I'm worried implementing it myself will just result in an additional set of hyperparameters to tune. [0] https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — mentioned by quotemstr

Sat, 06 Aug 2016 01:32:41 GMT

> Humans can design good features for the ML algorithm to use though For now. Articles like [1] make me wonder whether this current generation of programmers might be the last. [1] https://arxiv.org/abs/1606.04474

Paper and discussion history. A mention is not an endorsement.

Learning to learn by gradient descent by gradient descent — HN submission

Thu, 16 Jun 2016 15:35:13 GMT

Learning to learn by gradient descent by gradient descent

161 points; 40 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Learning to learn by gradient descent by gradient descent — HN submission

Thu, 16 Jun 2016 05:07:32 GMT

Learning to learn by gradient descent by gradient descent

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Learning to learn by gradient descent by gradient descent — HN submission

Wed, 15 Jun 2016 15:04:47 GMT

Learning to learn by gradient descent by gradient descent

6 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.