HackerPapers

arXiv:1509.08101cs.LG

Representation Benefits of Deep Feedforward Networks

Matus Telgarsky

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

This note provides a family of classification problems, indexed by a positive integer $k$, where all shallow networks with fewer than exponentially (in $k$) many nodes exhibit error at least $1/6$, whereas a deep network with 2 nodes in each of $2k$ layers achieves zero error, as does a recurrent network with 3 distinct nodes iterated $k$ times. The proof is elementary, and the networks are standard feedforward networks with ReLU (Rectified Linear Unit) nonlinearities.

arXiv comments
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.NENeural and Evolutionary Computing

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Oct 2015 to Oct 2015. Found in 1 HN submission from Oct 2015 to Oct 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Theoretical Motivations for Deep Learning 1 comment

    • There is a recent 5 page theoretical paper on this topic that I thought was pretty interesting, and it tackles both deep nets and recurrent nets: http://arxiv.org/abs/1509.08101 Here is the abstract: This note provides a family of classification problems, indexed by a positive…

      chriskananView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.