HackerPapers

arXiv:2210.08277cs.LG

Deep Differentiable Logic Gate Networks

Felix Petersen, Christian Borgelt, Hilde Kuehne, Oliver Deussen

Abstract

Recently, research has increasingly focused on developing efficient neural network architectures. In this work, we explore logic gate networks for machine learning tasks by learning combinations of logic gates. These networks comprise logic gates such as "AND" and "XOR", which allow for very fast execution. The difficulty in learning logic gate networks is that they are conventionally non-differentiable and therefore do not allow training with gradient descent. Thus, to allow for effective training, we propose differentiable logic gate networks, an architecture that combines real-valued logics and a continuously parameterized relaxation of the network. The resulting discretized logic gate networks achieve fast inference speeds, e.g., beyond a million images of MNIST per second on a single CPU core.

arXiv comments
Published at NeurIPS 2022
Journal reference
DOI
Subjects
cs.LGMachine Learning

On Hacker News

Linked by 7 identifiable commenters in 7 comments across 6 threads, from Feb 2023 to Nov 2025.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. TPUs vs. GPUs and why Google is positioned to win AI race in the long term 1 comment

    • SparseCores appear to be block-sparse as opposed to element-sparse. They use 8- and 16-wide vectors to compute. Here's another inference-efficient architecture where TPUs are useless: https://arxiv.org/pdf/2210.08277 There is no matrix-vector multiplication. Parameters are…

      theszView on HNParent context

  2. Weight-sparse transformers have interpretable circuits [pdf] 1 comment

    • Yes, you can relax logic gates into continuous versions which makes the system differentiable. An AND gate can be constructed with the function x*y and NOT by 1-x (on inputs in the range [0,1]. From there you can construct a NAND gate, which is universal and can be used to…

      calebhView on HNParent context

  3. Compiling a neural net to C for a speedup 2 comments

    • Differentiable Logic Gate Networks [0] are super interesting. However, I still don't like that the wiring is fixed initially rather than learned. I did some extremely rough research into doing learnable wirings [1], but couldn't get past even learning ~4-bit addition. [0]…

      enricozbView on HNParent context

    • I also worked a long time ago in recreating the original Deep Differentiable Logic Network paper [1], so I have a couple of additions to make. > I wanted to see if I could learn the wires in addition to the gates. I still think it’s possible, but it’s something I had to abandon…

      andy12_View on HNParent context

  4. GSoC'24: Differentiable Logic for Interactive Systems and Generative Music 1 comment

    • The original cited paper https://arxiv.org/abs/2210.08277 "Deep Differentiable Logic Gate Networks" struck me then as very clever, ultra-efficient/small, hardly any inductive bias or prior... But you have to wonder if it's able to scale reasonably well. Differentiating…

      gwernView on HNParent context

  5. Transformers on Chips 1 comment

    • Curious what approach you’re using. I did some work replicating this paper on an arty7 fpga: https://arxiv.org/abs/2210.08277 - any similarities?

      pmgView on HNParent context

  6. Logic Gate Simulator 1 comment

    • If you're into logic gates and deep learning: - Deep Differentiable Logic Gate Networks (https://arxiv.org/pdf/2210.08277.pdf) - Video presentation: Deep Differentiable Logic Gate Networks (https://www.youtube.com/watch?v=VulM891DhI4)

      dpflanView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.