Uniform convergence may be unable to explain generalization in deep learning — mentioned by cgdl

Wed, 19 Mar 2025 04:32:42 GMT

Yes, and that's the problem. What Zhang et al [2] showed convincingly in the Rethinking paper is that just focusing on the hypothesis space cannot be enough since the same hypothesis space fits real and random data so it's already too large. Therefore, these methods that focus on the hypothesis space have to talk about a bias in practice towards a better subspace, and that already requires studying the specific optimization algorithm in order to understand why it picks certain hypothesis over others in the space. But once you are ready to do that then algorithmic stability is enough. You…

Paper and discussion history. A mention is not an endorsement.

Uniform convergence may be unable to explain generalization in deep learning · HackerPapers discussions

Uniform convergence may be unable to explain generalization in deep learning — mentioned by cgdl