Local minima in training of neural networks — mentioned by chillee

Wed, 27 Dec 2017 00:57:43 GMT

That's why I said high dimensional neural networks. There's been a lot of literature explaining why local minima aren't a problem in very high dimension loss surfaces. Check any of the literature on this subject: https://arxiv.org/abs/1611.06310v2 https://arxiv.org/abs/1406.2572 Local minima are something that people thought was gonna be a problem, especially back in the 2000s. They played around with small neural nets on toy examples such as yours, and thought it was intractable. It's the entire reason why neural nets fell out of the fashion in the early 2000s, and people moved towards…

Paper and discussion history. A mention is not an endorsement.

Local minima in training of neural networks · HackerPapers discussions

Local minima in training of neural networks — mentioned by chillee