Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle · HackerPapers discussions

Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle — HN submission

Wed, 23 Apr 2025 23:09:34 GMT

Double Descent Demystified: size of smallest non-zero singular value of X

8 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle — HN submission

Sun, 15 Sep 2024 18:48:07 GMT

Double Descent Demystified

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle — mentioned by ds_opseeker

Wed, 31 Jul 2024 09:39:56 GMT

[Submitted on 24 Mar 2023] Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle Rylan Schaeffer, Mikail Khona, Zachary Robertson, Akhilan Boopathy, Kateryna Pistunova, Jason W. Rocks, Ila Rani Fiete, Oluwasanmi Koyejo https://arxiv.org/abs/2303.14151 Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever larger into the highly overparameterized (data undersampled) regime. This drop in test error flies against classical…

Paper and discussion history. A mention is not an endorsement.

Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle — mentioned by iaseiadit

Sat, 02 Mar 2024 21:30:00 GMT

Not an expert, but this paper explores double descent with simple models. The interpretation there: when you extend into the overparameterized regime, that permits optimization towards small-norm weights, which generalize well again. Does that explain DD generally? Does it apply to other models (e.g. DNNs)? https://arxiv.org/pdf/2303.14151.pdf

Paper and discussion history. A mention is not an endorsement.

Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle — mentioned by canjobear

Wed, 26 Apr 2023 06:11:43 GMT

The formal claim is in Appendix A here: https://arxiv.org/pdf/2303.14151.pdf

Paper and discussion history. A mention is not an endorsement.