HackerPapers

arXiv:1412.7149cs.LG

Deep Fried Convnets

Zichao Yang, Marcin Moczulski, Misha Denil, Nando de Freitas, Alex Smola, Le Song, Ziyu Wang

Abstract

The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters. Reducing the number of parameters while preserving essentially the same predictive performance is critically important for operating deep neural networks in memory constrained environments such as GPUs or embedded devices. In this paper we show how kernel methods, in particular a single Fastfood layer, can be used to replace all fully connected layers in a deep convolutional neural network. This novel Fastfood layer is also end-to-end trainable in conjunction with convolutional layers, allowing us to combine them into a new architecture, named deep fried convolutional networks, which substantially reduces the memory footprint of convolutional networks trained on MNIST and ImageNet with no drop in predictive performance.

arXiv comments
svd experiments included
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.NENeural and Evolutionary Computing stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 3 identifiable commenters in 4 comments across 4 threads, from Jun 2015 to Feb 2018.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. Lessons from Optics, the Other Deep Learning 1 comment

    • This reminds me of ACDC, a Deep-fried Convnets-like[0] approach by some NVIDIA employees. [1] See section 1.1, where they state they could perform the operation in analog. [0]: https://arxiv.org/abs/1412.7149 [1]: https://arxiv.org/pdf/1511.05946

      stochastic_monkView on HNParent context

  2. The often-overlooked random forest kernel 1 comment

    • Very interesting; I've ignored random forests more than I should. Thank you! I won't comment on your methods except to say that comparing classification over linear PCA against a kernel PCA in linearly unseparable data isn't exactly fair, and I think that providing an SVM…

      stochastic_monkView on HNParent context

  3. Deep-Fried Data 1 comment

    • Rather hilariously, deep frying is already a term of art in ML, of course in a radically different setting. Deep fried convnets (https://arxiv.org/abs/1412.7149).

      curuinorView on HNParent context

  4. What's Wrong with Deep Learning? 1 comment

    • Actually, state of the art speech recognition has switched over to having a Recursive Neural Network directly run over the audio input. Take a look at the paper at http://arxiv.org/abs/1412.5567 and http://usa.baidu.com/deep-speech-lessons-from-deep-learning/ However combining…

      jhartmannView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.