Deep Learning is Not So Mysterious or Different · HackerPapers discussions

Deep Learning is Not So Mysterious or Different — mentioned by robrenaud

Tue, 22 Sep 2026 17:59:38 GMT

Look up the double descent phenomenon. Large models with lots of data generalize rather than overfit. Also see the argument around figure 3 here. https://arxiv.org/pdf/2503.02113

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Not So Mysterious or Different — HN submission

Sun, 30 Aug 2026 09:06:15 GMT

Deep Learning Is Not So Mysterious or Different

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Deep Learning is Not So Mysterious or Different — mentioned by Macuyiko

Thu, 07 May 2026 10:55:06 GMT

Reminded me strongly of the paper "Deep Learning is Not So Mysterious or Different" from a year ago: https://arxiv.org/abs/2503.02113

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Not So Mysterious or Different — mentioned by ainch

Sun, 26 Apr 2026 11:54:14 GMT

Check out Andrew Gordon Wilson's excellent paper "Deep Learning is Not so Mysterious or Different" for a discussion of the ways in which existing learning theory does and doesn't work neural nets. https://arxiv.org/pdf/2503.02113

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Not So Mysterious or Different — mentioned by hodgehog11

Thu, 14 Aug 2025 14:10:39 GMT

Are you asking about open problems? Here are a couple: There are strong arguments that deep learning generalization (and robustness in a slightly different sense) can be successfully explained in terms of PAC-Bayes theory, for example: https://arxiv.org/abs/2503.02113 This requires a strong understanding of the appropriate prior, which effectively quantifies implicit bias in the training setup / architecture setup. Early analyses suggest the bias is that of a minimal gradient norm (https://arxiv.org/pdf/2101.12176) but this is almost certainly not the case…

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Not So Mysterious or Different — mentioned by thesz

Tue, 22 Apr 2025 21:22:32 GMT

https://arxiv.org/pdf/2503.02113 This paper shows that polynomials show most features of deep neural nets, including double descent and ability to memorize entire dataset. It connects dots there - polynomials there are regularized to be as simple as possible and author argues that hundredths of billions of parameters in modern neural networks work as a regularizers too, they attenuate decisions that "too risky." I really enjoyed that paper, a gem that puts light everywhere.

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Not So Mysterious or Different — HN submission

Mon, 17 Mar 2025 16:47:02 GMT

Deep Learning Is Not So Mysterious or Different

485 points; 126 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Deep Learning is Not So Mysterious or Different — HN submission

Fri, 07 Mar 2025 17:11:27 GMT

Deep Learning Is Not So Mysterious or Different

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.