https://hackerpaper.kaipfister.de/p/2412.09810/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=47331048urn:hackerpapers:mention:2412.09810:47331048Wed, 11 Mar 2026 02:06:11 GMT<p>You literally can do a kind of model PCA, using the Hessian (matrix of second derivatives of the loss function w/r/t the parameters, aka the local curvature of the loss landscape), and diagonalizing. These eigenvectors and eigenvalues (the spectrum of the Hessian) tend to be power-law distributed in just about every deep NN you can think of [1]. That is, there are a few &quot;really important&quot; (highly curved) dimensions in parameter space (the top eigenvectors) which control the model&#x27;s performance (the loss function). Conversely, there are very many &quot;unimportant&quot;/low curvature dimensions in the…</p><p><a href="https://hackerpaper.kaipfister.de/p/2412.09810/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG