A Theory of Generalization in Deep Learning · HackerPapers discussions

A Theory of Generalization in Deep Learning — mentioned by T-A

Thu, 07 May 2026 08:59:02 GMT

Did you also know about this? Lastly, we derive an exact population-risk objective from a single training run with no validation data, for any architecture, loss, or optimizer, and prove that it measures precisely the noise in the signal channel. This objective reduces in practice to an SNR preconditioner on top of Adam, adding one state vector at no extra cost; it accelerates grokking by 5x, suppresses memorization in PINNs and implicit neural representations, and improves DPO fine-tuning under noisy preferences while staying 3x closer to the reference policy. [1] [1]…

Paper and discussion history. A mention is not an endorsement.

A Theory of Generalization in Deep Learning — mentioned by r0ze-at-hn

Thu, 07 May 2026 08:14:14 GMT

Linking to the paper: https://arxiv.org/pdf/2605.01172 which is also a fantastic read, the application to deep learning is good. It does a lot of cross-mapping and highlighting a bunch of old stuff that is named differently in this paper and worth calling out for those with those backgrounds: "Cumulative Dissipation Gramian" Ws = Observability Gramian (from Control Theory). For example the spectral cutoff is exactly the Hankel singular value truncation from model reduction. "Signal Channel" / "Reservoir" is Controllable/Observable vs. Uncontrollable/Unobservable Subspaces. Using…

Paper and discussion history. A mention is not an endorsement.

A Theory of Generalization in Deep Learning — mentioned by vessenes

Thu, 07 May 2026 00:19:48 GMT

So, this is either the paper of the year, or ... definitely not the paper of the year. https://arxiv.org/pdf/2605.01172 is the current version. The money graphs are page 8 and on where they show (some weirdly thick) line charts with loss results reached in roughly 1/5 the number of steps that Adam takes, just what the blog post mentions. They also claim holding back test data is not needed, also with more graphs. I'm not an ML scientist, and I did not attempt to seriously parse the math. It reads to me as something precisely in that liminal space some math papers do where there's enough new…

Paper and discussion history. A mention is not an endorsement.

A Theory of Generalization in Deep Learning — mentioned by ks2048

Wed, 06 May 2026 19:45:51 GMT

The relevant paper: "A Theory of Generalization in Deep Learning". https://arxiv.org/abs/2605.01172

Paper and discussion history. A mention is not an endorsement.

A Theory of Generalization in Deep Learning — HN submission

Tue, 05 May 2026 18:31:23 GMT

A Theory of Generalization in Deep Learning

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.