Surprises in High-Dimensional Ridgeless Least Squares Interpolation · HackerPapers discussions

Surprises in High-Dimensional Ridgeless Least Squares Interpolation — mentioned by hodgehog11

Sun, 26 Apr 2026 06:58:15 GMT

Hastie was actually lead author of an excellent paper that discusses the underlying phenomenon in the context of least-squares linear regression: https://arxiv.org/abs/1903.08560 It really isn't so mysterious once you begin to examine how the rule of thumb for the bias-variance tradeoff (remember that it is the relationship with model size that is curious, not the tradeoff itself) came to be. The easiest ways to arrive at this rule are through an information criterion like the AIC or BIC, where the model size appears in the penalty term for the log-likelihood. These criteria have a bunch of…

Paper and discussion history. A mention is not an endorsement.

Surprises in High-Dimensional Ridgeless Least Squares Interpolation — mentioned by cs702

Sun, 26 Jul 2020 14:55:00 GMT

> ... all DNNs face the issue of overfitting as they learn, which is when performance on one data set increases but the network's performance fails to generalize (often measured by the divergence of performance on training vs testing data sets). This ubiquitous problem in DNNs is often solved by experimenters via "noise injections" in the form of noisy or corrupted inputs. The goal of this paper is to argue that the brain faces a similar challenge of overfitting, and that nightly dreams evolved to combat the brain's overfitting during its daily learning. Actually, there's compelling evidence…

Paper and discussion history. A mention is not an endorsement.