Two models of double descent for weak features — mentioned by seesawtron

Sun, 26 Jul 2020 15:26:59 GMT

Most of the DL models we use are already above the "interpolation threshold" because they use millions of parameters which implies that they are already being trained in the "double descent" regime. Those papers and more [0] argue to explain why using millions of parameters still doesn't overfit our models that are trained on much less data (opposing the traditional Vapnik machine learning argument that your model paramters should not exceed number of data points otherwise you see overfit) because the models still manage to perform well on test data. In this paper, I think the authors are not…

Paper and discussion history. A mention is not an endorsement.

Two models of double descent for weak features · HackerPapers discussions

Two models of double descent for weak features — mentioned by seesawtron