Every Model Learned by Gradient Descent Is Approximately a Kernel Machine · HackerPapers discussions

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — HN submission

Sun, 27 Jul 2025 19:04:23 GMT

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by babel_

Thu, 28 Mar 2024 13:59:52 GMT

So, from the perspective I have within the subfield I work in, explainable AI (XAI), we're seeing a bunch of fascinating developments. First, as you mentioned, Rudin continues to prove that the reason for using AI/ML is that we don't understand the problem well enough; otherwise we wouldn't even think to use it! So, pushing our focus to better understand the problem, and then levy ML concepts and techniques (including "classical AI" and statistical learning), we're able to make something that not only outperforms some state-of-the-art in most metrics, but often even is much less resource…

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — HN submission

Sun, 25 Feb 2024 01:25:02 GMT

Every model learned by gradient descent is approximately a kernel machine (2020)

176 points; 136 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by RamblingCTO

Tue, 07 Nov 2023 19:18:44 GMT

*approximately https://arxiv.org/abs/2012.00152

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by Joschkabraun

Sat, 28 Oct 2023 10:30:38 GMT

Reminds me of "Every Model Learned by Gradient Descent Is Approximately a Kernel Machine" by Pedro Domingos: https://arxiv.org/abs/2012.00152

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by visarga

Thu, 10 Aug 2023 17:04:41 GMT

> Statistical learning can typically be phrased in terms of k nearest neighbours We have suspected that neural nets are a kind of kNN. Here's a paper: Every Model Learned by Gradient Descent Is Approximately a Kernel Machine https://arxiv.org/abs/2012.00152

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — HN submission

Tue, 10 Jan 2023 11:47:05 GMT

Every Model Learned By Gradient Descent Is Approximately A Kernel Machine (2020)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by visarga

Sat, 24 Sep 2022 10:53:01 GMT

Reminds me of Pedro Domingos's paper - Every Model Learned by Gradient Descent Is Approximately a Kernel Machine https://arxiv.org/abs/2012.00152

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by FeepingCreature

Wed, 06 Apr 2022 03:46:10 GMT

> All GD learners are interpolators (cf. https://arxiv.org/abs/2012.00152) , If the paper is substantially correct I concede the point. But what I've read of reactions leads me to believe the conclusion is overstated. Regarding compression vs intelligence, I already believe that intelligence, even human intelligence, is largely a matter of compressing data. Regarding "knowledge is deterministic", ignoring the fact that it's not even deterministic in humans, so long as GPT can instantiate agents I consider the question of whether it "is" an agent academic. If GPT can operate over W_m and H_n…

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by mjburgess

Tue, 05 Apr 2022 14:01:24 GMT

All GD learners are interpolators (cf. https://arxiv.org/abs/2012.00152) we also know theyre exponential in parameter count ( cf. https://www.researchgate.net/figure/Number-of-parameters-ie-... ) > If GPT is not exponential, then the m/p distinction falls apart. Yes, I think if you have a system which implements QAWH with a similar compelxity to a known intelligent system -- at that point I have no empirical issues. I think, at that point, you have a workiung system. We then ask if it is thinking about anything, and I think that'd be an open question as to how its implemented. I dont think…

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by YeGoblynQueenne

Thu, 10 Mar 2022 17:37:39 GMT

Sorry, my fault: I don't mean that PAC-Learnability means that neural nets memorise their training instances. That's more my interpretation of their observed behaviour, if you like. What I meant was that PAC-Learnability doesn't assume any ability like reasoning, and really no other ability than er, PAC-Learnability. There's a debate, of course. I like to point to Domingos' paper: Every Model Learned by Gradient Descent Is Approximately a Kernel Machine https://arxiv.org/abs/2012.00152 With the full understanding that it's just one paper.

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by homerowilson

Tue, 12 Oct 2021 13:42:26 GMT

A nice, short, recent paper along these lines (not mentioned in the article) is "Every Model Learned by Gradient Descent Is Approximately a Kernel Machine": https://arxiv.org/abs/2012.00152

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — mentioned by antipaul

Sun, 27 Dec 2020 19:41:11 GMT

Thanks for the reference. Here it is: “We show, however, that deep networks learned by the standard gradient descent algorithm are in fact mathematically approximately equivalent to kernel machines, a learning method that simply memorizes the data and uses it directly for prediction via a similarity function (the kernel)“ https://arxiv.org/abs/2012.00152

Paper and discussion history. A mention is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — HN submission

Sat, 05 Dec 2020 14:40:31 GMT

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine

406 points; 107 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — HN submission

Fri, 04 Dec 2020 13:05:25 GMT

Deep learning networks are approximately kernel machines

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine — HN submission

Thu, 03 Dec 2020 00:44:57 GMT

Every Model Learned by Gradient Descent Is Approximately a Kernel Machine

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.