Fisher-Rao Metric, Geometry, and Complexity of Neural Networks — mentioned by chillee

Mon, 18 Dec 2017 22:20:36 GMT

Sorry, my last sentence > Otherwise, models can achieve very good generalization. was probably unclear. When we talk about generalization, we typically have to assume a lack of distributional shift. Otherwise, if the "truly unseen data" really is drawn from a different distribution (say one where all cats are now called "dogs" and all dogs are now called "cats"), we can get arbitrarily poor performance. As for the flat minima part, lemme pull up some papers from my notes. https://arxiv.org/abs/1609.04836 showed larger batch sizes lead to sharper minima. > ... and as is well known, sharp…

Paper and discussion history. A mention is not an endorsement.

Fisher-Rao Metric, Geometry, and Complexity of Neural Networks · HackerPapers discussions

Fisher-Rao Metric, Geometry, and Complexity of Neural Networks — mentioned by chillee