HackerPapers

arXiv:1603.05279cs.CV

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, Ali Farhadi

Abstract

We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32x memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58x faster convolutional operations and 32x memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is only 2.9% less than the full-precision AlexNet (in top-1 measure). We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than 16% in top-1 accuracy.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition

On Hacker News

Linked by 7 identifiable commenters in 8 comments across 7 threads, from Apr 2016 to Nov 2024. Found in 1 HN submission from Mar 2016 to Mar 2016. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments7 threads

  1. SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup 1 comment

    • Exactly as xnornet was doing way back in 2016 - shadow 32bit weights, quantized to 1 bit during the forward pass. https://arxiv.org/abs/1603.05279 I personally have a pretty negative opinion of the bitnet paper.

      buildbotView on HNParent context

  2. Shrinking Machine Learning Models for Offline Use 1 comment

    • There are low-bit networks as well: https://arxiv.org/abs/1603.05279 That technology is now a spinoff of AI2 and UW: https://www.xnor.ai/

      QworgView on HNParent context

  3. Intel Gets Serious About Neuromorphic, Cognitive Computing Future 2 comments

    • The article misses the core issue: neural network architectures are still in flux. "Neuromorphic" chips are hardwired to one architecture, which makes them power efficient but less flexible. When designs are more stable, such chips could be more practical. Meanwhile, upcoming…

      paulsutterView on HNParent context

    • The reality is that TrueNorth can use CNNs and be trained with backprop to achieve good performance with networks like ImageNet despite LeCun's comments [0, 1] LeCun's primary critique is that binary wont work; "to get good results on a task like ImageNet you need about 8 bit of…

      emcqView on HNParent context

  4. XNOR.ai frees AI from the prison of the supercomputer 1 comment

    • XNOR-Net is not currently part of TensorFlow, although there is a feature request to support it. [0] The main difference is that XNOR-Net uses 1 bit quantized weights and XNOR+popcount to approximate the dot product of convolution, which can be implemented very efficiently by…

      emcqView on HNParent context

  5. TensorFlow machine learning now optimized for the Snapdragon 835 and Hexagon 682 1 comment

    • Nice, looks like about 10x speed up for this classification task. I think there are big gains to be made in lower precision inference too. Lots of people doing interesting work in that area, check out these guys - https://xnor.ai/ https://arxiv.org/abs/1603.05279

      dharma1View on HNParent context

  6. Notes from the fourth RISC-V workshop 1 comment

    • This is incorrect. With custom ASIC that implements ultra-wide low precision SIMD arithmetic (f16, i16, i8) it is possible to squeeze an order of magnitude or more speedup over conventional GPGPUs using same area and power. GPGPUs have to have big f32 (and even f64!) FPUs, and…

      sapphireblueView on HNParent context

  7. Real-Time Texture Synthesis with Markovian Generative Adversarial Networks 1 comment

    • Since it seems this paper's primary focus is on performance, it'd be interesting to see how this technique stacks up against one of those fancy new binary networks (e.g. http://arxiv.org/abs/1603.05279)

      ericfloView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.