Hopfield Networks is All You Need · HackerPapers discussions

Hopfield Networks is All You Need — mentioned by vatsachak

Thu, 08 Jan 2026 03:01:09 GMT

https://arxiv.org/abs/2008.02217 They derive Q, K, V as a continuous analog of a hopfield network

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — HN submission

Mon, 04 Aug 2025 18:34:22 GMT

Hopfield Networks Is All You Need (2020)

https://github.com/ml-jku/hopfield-layers

41 points; 6 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Hopfield Networks is All You Need — mentioned by korbip

Sat, 15 Feb 2025 03:04:16 GMT

There is a LOT of effort in the research community currently: 1. Improving the Self-Attention in the Transformer as is, keeping the quadratic complexity, which has some theoretical advantage in principle[1]: The most hyped one probably DeepSeek's Multi-Latent Attention[15], which kind of is Attention still - but also somehow different. 2. Linear RNNs: This starts from Linear Attention[2], DeltaNet[3], RKWV[4], Retention[5], Gated Linear Attention[6], Mamba[7], Griffin[8], Based[9], xLSTM[10], TTT[11], Gated DeltaNet[12], Titans[13]. They all have an update like: C_{t} = F_{t} C_{t-1} + i_{t}…

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by igleria

Fri, 14 Feb 2025 11:41:20 GMT

This immediately makes my mind bring up Hopfield networks https://arxiv.org/abs/2008.02217 when I worked with them circa 2012 they were practically toys. Maybe we are in a better place now?

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by adamnemecek

Mon, 04 Mar 2024 02:26:55 GMT

This is not surprising considering the paper "Hopfield Networks is All You Need" https://arxiv.org/abs/2008.02217. Hopfield networks learn via Hebbian learning.

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by whimsicalism

Sun, 23 Jul 2023 05:26:41 GMT

> 1) there are a lot of papers claiming to be the successor to the Transformer, and not all of them are cited; e.g., the MetaFormer is missing https://arxiv.org/abs/2111.11418. Another candidate that wasn't compares against (or at least argued why it wouldn't make sense to compare against) are the Hopfield Networks https://arxiv.org/abs/2008.02217. Neither of those papers are NLP applicable? And I think it's perfectly fair to focus on the alternatives (ie. like H3 and RWKV) that have been able to scale up to LLM levels and perplexity, which neither of the alternatives you mention have. Should…

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by sfriedr

Sun, 23 Jul 2023 04:55:58 GMT

I only spent a few minutes skimming thr paper, but: 1) there are a lot of papers claiming to be the successor to the Transformer, and not all of them are cited; e.g., the MetaFormer is missing https://arxiv.org/abs/2111.11418. Another candidate that wasn't compares against (or at least argued why it wouldn't make sense to compare against) are the Hopfield Networks https://arxiv.org/abs/2008.02217. So until a more solid Related Work section is written (their section is actually called "Relation to and Differences from Previous Methods") I reserve the right to be skeptical whether their model…

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by macrolocal

Wed, 14 Sep 2022 16:52:04 GMT

More accessible references: https://mcbal.github.io/post/an-energy-based-perspective-on-... (Modern continuous Hopfield networks section) https://arxiv.org/abs/2008.02217 Note that the connection to Hebbian learning hinges on the softmax function, in particular its exponential!

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by flowtheorist

Sun, 06 Feb 2022 10:39:18 GMT

It's definitely in a gray area because the AI models are essentially compression engines that encode the code samples/data into the weights of the matrices that represent the ML model and then "uncompress" it to serve queries. I think it would be easy to argue that a compressed data set no matter how illegible would need to conform to the same license as the data set it was encoding but I don't think any lawyer is smart enough to make that case. So at the moment it remains a very convenient loophole for companies that have enough compute to mangle the data set beyond recognition and then use…

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — mentioned by sillysaurusx

Thu, 05 Aug 2021 00:32:59 GMT

> If I recall correctly, it was a Hopfield thingie, so I'm pretty off topic here. Actually, you’re spot on. Hopfield networks were once popular, and are now popular again: https://arxiv.org/abs/2008.02217 (Hopularity leads to popularity, as they say.) I wish there was some way to track down that spreadsheet. :) It’d be neat to read. Thanks for sharing.

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — HN submission

Wed, 16 Jun 2021 18:39:54 GMT

Transformers Connected to Hopfield Networks

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Hopfield Networks is All You Need — mentioned by KhoomeiK

Tue, 08 Dec 2020 21:24:03 GMT

Discovering Symbolic Models from Deep Learning with Inductive Biases [1] trains graph neural nets on astrophysical phenomena and then performs symbolic regression to generate algebraic formulae to elegantly model the phenomena in a classical physics framework. It's largely gone under the radar but has pretty interesting implications for NLP and language theory in my opinion. Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures [2] applies DFA, an approach to training neural nets without backprop, to modern architectures like the Transformer. It does surprisingly…

Paper and discussion history. A mention is not an endorsement.

Hopfield Networks is All You Need — HN submission

Fri, 28 Aug 2020 18:28:33 GMT

Hopfield Networks Is All You Need

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Hopfield Networks is All You Need — HN submission

Wed, 26 Aug 2020 09:34:02 GMT

Hopfield Networks Is All You Need

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Hopfield Networks is All You Need — HN submission

Fri, 14 Aug 2020 14:51:21 GMT

Hopfield Networks Is All You Need

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Hopfield Networks is All You Need — HN submission

Thu, 06 Aug 2020 09:29:25 GMT

Hopfield Networks Is All You Need

4 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.