This paper provides theoretical insights into why and how deep learning can generalize well, despite its large capacity, complexity, possible algorithmic instability, nonrobustness, and sharp minima, responding to an open question in the literature. We also discuss approaches to provide non-vacuous generalization guarantees for deep learning. Based on theoretical observations, we propose new open problems and discuss the limitations of our results.
arXiv comments
Published by Cambridge University Press. BibTeX of this paper is available at: https://people.csail.mit.edu/kawaguch/bibtex.html
Journal reference
In Mathematical Aspects of Deep Learning. Cambridge University Press, 2022
Linked by 4 identifiable commenters in 5 comments across 5 threads, from Oct 2017 to Jun 2021. Found in 1 HN submission from Oct 2017 to Oct 2017. Open submission discussions.
The AI most certainly does not lack the ability to generalize. Not as well as humans, but generalization is the key interesting result in deep learning, leading to papers like this one: https://arxiv.org/abs/1710.05468 The ability to generalize actually seems to keep increasing…
I haven't read this paper yet so I can't speak to it's quality but it appears to be addressing the same questions in this post. Bengio is a coauthor so maybe that's a good sign . Here's the abstract. This paper provides theoretical insights into why and how deep learning can…
https://arxiv.org/abs/1710.05468 was an interesting paper that came out recently. It showed that large CNN models which have far greater capacity than the data they are shown (and could have memorized it) still tend to learn very good generalizable minima. See proposition 1: (i)…
We know no more about the theory of LSTM networks than we do about ordinary NNs. Perhaps, but we do know a lot more about how (deep) NNs work than we did 40 years ago. Take for example: Generalization in Deep Learning[1] which gives a theoretical grounding of generalization.…
"boys" Not sure what this was supposed to mean? Yes, I think Fei Fei Li's datasets are much better tests than MNIST if that is what you were getting at? I don't see any of these people submitting MNIST to NIPS in 2017 None of them submitted things as entirely new and different…
The AI most certainly does not lack the ability to generalize. Not as well as humans, but generalization is the key interesting result in deep learning, leading to papers like this one: https://arxiv.org/abs/1710.05468 The ability to generalize actually seems to keep increasing…
gradysView on HNParent context