Adam: A Method for Stochastic Optimization · HackerPapers discussions

Adam: A Method for Stochastic Optimization — mentioned by brosco

Sat, 18 Oct 2025 06:40:25 GMT

One of OpenAI's founding team members developed Adam [0] well before it was flashy and profitable. It's not like nobody is out there trying to develop new algorithms. The reality is that there are some great, mature solvers out there that work well enough for most cases. And while it might be possible to eke out more performance in specific problems, it would be very hard to beat existing solvers in general. Theoretical developments like this, while interesting on their own, don't really contribute much to day-to-day users of linear programming. A lot of smart people have worked very hard to…

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by cherryteastain

Tue, 18 Mar 2025 12:23:51 GMT

As a (former) reviewer at 5 journals, I disagree first and foremost with the notion that > We need some process to gate-keep. Journals, when print was the medium through which academic research was disseminated, had to gatekeep because there were practical considerations regarding how many articles they could put in each issue. With online repositories like arxiv, this is hardly a concern anymore. Someone putting a crap article on arxiv does not hurt anyone else, and I'm saying this as a person who recommended tons of articles to be rejected because they had atrocious grammar/spelling issues.…

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by modeless

Tue, 01 Oct 2024 20:10:27 GMT

Kingma is most notable for writing one of the most cited papers in AI. Actually one of the most cited scientific papers ever published, right up there with the transformers paper if not higher. "Adam: A Method for Stochastic Optimization" https://arxiv.org/abs/1412.6980 I remain astonished that Adam continues to be the most widely used optimizer in AI 10 years later. So many contenders have failed to replace it.

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by GistNoesis

Thu, 09 May 2024 12:25:59 GMT

Acquisitions, like for DeepMind is usually a way to hire talent. It can make sense when the technology is new and getting a few year of lead time on what is going to be a growing market may make some financial sense. In this specific xLSTM case, the industry has matured, they are just one among many (Mamba, S3Ms, transformers-variants... ), they have already been sitting on it for at least 6 months, I don't see what their play is. An other case study that's probably interesting, are the authors of the Adam Paper, https://arxiv.org/abs/1412.6980 , (Awarded "2020: The Adam optimization paper is…

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by troelsSteegin

Tue, 18 Jul 2023 15:53:39 GMT

"The name Adam is derived from adaptive moment estimation.", https://arxiv.org/abs/1412.6980

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by lostmsu

Fri, 03 Feb 2023 19:33:48 GMT

Frankly, one does not need this paper to get towards the AI. Adam the optimization algo you might need (and even there I am not sure). And it is very readable. The fact that this particular proof of Adam's convergence is complicated is largely irrelevant. https://arxiv.org/pdf/1412.6980.pdf

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by aqsalose

Tue, 03 Oct 2017 18:04:22 GMT

This is the PhD thesis of Durk Kingma [1]. Reading this right now; appears to be a well-written introduction to stuff variational autoencoders [2] and ADAM [3]. [1] http://dpkingma.com/ [2] http://arxiv.org/abs/1312.6114 [3] http://arxiv.org/abs/1412.6980

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — mentioned by brandonb

Mon, 02 Jan 2017 20:41:08 GMT

This seems like a sincere comment, so in case it's helpful, here's my interpretation of why others may be downvoting it. Deep learning practitioners are aware that they're using non-convex optimization, stochastic gradient descent, and so on. In fact, those topics are core to modern deep learning research and explicitly acknowledged in most published research: LSTMs were invented to solve the vanishing gradient problem, Restricted Boltzmann Machines were used as a pre-training step to avoid local minima, and optimizers like ADAM have explicit guarantees about things like convergence. You may…

Paper and discussion history. A mention is not an endorsement.

Adam: A Method for Stochastic Optimization — HN submission

Tue, 13 Oct 2015 19:42:03 GMT

Adam's method for stochastic optimization

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.