HackerPapers

arXiv:1805.08671stat.ML

Adding One Neuron Can Eliminate All Bad Local Minima

Shiyu Liang, Ruoyu Sun, Jason D. Lee, R. Srikant

HN submission. Found in a submission link or body. Submitters do not count as commenters in the main ranking; a submission is not an endorsement.

Abstract

One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection to the output, or one special neuron per layer, every local minimum is a global minimum.

arXiv comments
Journal reference
DOI
Subjects
stat.MLMachine Learning (Statistics) cs.LGMachine Learning

On Hacker News

No comments linking this paper were found in the collected sources. Found in 1 HN submission from May 2018 to May 2018. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments0 threads

No comments linking this paper were found in the collected sources. Open submission discussions.

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.