Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization — mentioned by blt

Wed, 08 May 2024 17:58:12 GMT

[1] Was an empirical paper that inspired much theoretical follow-up. [2] Is one such follow-up, and the references therein should point to many of the other key works in the years between. [3] Introduces the neural tangent kernel (NTK), a theoretical tool used in much of this work. (Not everyone agrees that reliance on NTK is the right way towards long-term theoretical progress.) [4] Is a more recent paper I haven't read yet that goes into more detail on interpolation. Its authors were well known in more "clean" parts of ML theory (e.g. bandits) and recently began studying deep learning.…

Paper and discussion history. A mention is not an endorsement.

Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization · HackerPapers discussions

Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization — mentioned by blt