Do Deep Nets Really Need to be Deep? · HackerPapers discussions

Do Deep Nets Really Need to be Deep? — mentioned by rcpt

Thu, 02 Jun 2016 06:15:39 GMT

Good iclr paper on this "Do Deep Nets Really Need to be Deep?" http://arxiv.org/abs/1312.6184

Paper and discussion history. A mention is not an endorsement.

Do Deep Nets Really Need to be Deep? — mentioned by nharada

Thu, 15 Jan 2015 01:08:45 GMT

Deep neural networks are quite powerful because they don't require someone to design the network specific to the task. Large deep nets will generally be able to learn feature representations by themselves, which makes them extremely powerful for tasks where we aren't quite sure of exactly what features we want. However, deep nets require large amounts of training data and computational power. The fairly recent widespread adoption of general purpose GPUs has allowed much faster training. Combine this with the popularity of "big data", and you've got a perfect storm for deep neural nets. Of…

Paper and discussion history. A mention is not an endorsement.

Do Deep Nets Really Need to be Deep? — mentioned by Sniffnoy

Sat, 04 Oct 2014 21:32:58 GMT

A note -- if you're linking to arXiv, it's better to link to the abstract (http://arxiv.org/abs/1312.6184) rather than directly to the PDF. From the abstract, one can easily click through to the PDF; not so the reverse. And the abstract allows one to do things like see different versions of the paper, search for other things by the same authors, etc.

Paper and discussion history. A mention is not an endorsement.

Do Deep Nets Really Need to be Deep? — HN submission

Sat, 04 Oct 2014 01:41:07 GMT

Do Deep Nets Really Need to be Deep? [pdf]

126 points; 29 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Do Deep Nets Really Need to be Deep? — mentioned by robrenaud

Wed, 10 Sep 2014 05:53:56 GMT

Have you read Rich Cuarana's work on approximating the behavior of deep nets with single hidden layer networks? http://arxiv.org/pdf/1312.6184.pdf From that paper, it seems like it's just that deep nets are easier to train to find state of the art solutions than shallow nets. It's not that shallow nets are inherently incapable of performing as well. Shallow nets can mimic/approximate a given well trained deep net and preserve almost all of the accuracy. So it's not the case that a good solution to the task doesn't exist in the set of hypothesizes spanned by sized shallow net, it's just that…

Paper and discussion history. A mention is not an endorsement.

Do Deep Nets Really Need to be Deep? — mentioned by kastnerkyle

Fri, 08 Aug 2014 11:34:42 GMT

I find it interesting that no group (to my knowledge) has tried something similar to [Do Deep Networks Need to Be Deep?](http://arxiv.org/abs/1312.6184) for ImageNet scale networks. There have been several results which show that the knowledge learned in larger networks can be compressed and approximated using small or even single layer nets. Extreme learning machines (ELM) can be seen as another aspect of this. There have also been interesting results in the "kernelization" of convnets [from Julian Mairal and co.](http://arxiv.org/abs/1406.3332) that, accompanied by the stong crossover…

Paper and discussion history. A mention is not an endorsement.

Do Deep Nets Really Need to be Deep? — HN submission

Mon, 13 Jan 2014 17:14:40 GMT

Do Deep Nets Really Need to be Deep?

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Do Deep Nets Really Need to be Deep? — HN submission

Fri, 03 Jan 2014 09:33:32 GMT

Shallow neural networks as good as Deep Neural Networks

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.