Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 · HackerPapers discussions

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by bjornsing

Fri, 29 Mar 2024 18:40:37 GMT

First version of BNN was submitted to ArXiv a month before XNOR-Net: https://arxiv.org/abs/1602.02830

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by oxxoxoxooo

Wed, 28 Feb 2024 19:23:46 GMT

Prior art: Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 https://arxiv.org/abs/1602.02830 Ternary Neural Networks for Resource-Efficient AI Applications https://arxiv.org/abs/1609.00222

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by quotemstr

Thu, 21 Nov 2019 14:03:22 GMT

> i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/articles/10778 Thanks for the link. Artificial neural networks all the way down to binary weights [1] although this approach doesn't seem like the most efficient one. It's interesting how we're still seeing a ton of variability in ML architectures: it suggests we haven't stumbled on the right area yet. It reminds me how early aviation had a huge diversity of aircraft plans, but now, after a lot of optimization, we've settled on that one standard airliner shape everyone…

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by iandanforth

Sat, 20 Oct 2018 23:40:07 GMT

>> NNs are not binary -- they're fundamentally analog That's incorrect. https://arxiv.org/abs/1602.02830

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by darkmighty

Fri, 29 Sep 2017 21:32:24 GMT

It's also possible to binarize DNNs to use the faster bitwise operations https://arxiv.org/pdf/1602.02830.pdf

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by emcq

Sat, 11 Feb 2017 21:27:32 GMT

The reality is that TrueNorth can use CNNs and be trained with backprop to achieve good performance with networks like ImageNet despite LeCun's comments [0, 1] LeCun's primary critique is that binary wont work; "to get good results on a task like ImageNet you need about 8 bit of precision on the neuron states". There was no evidence for his claim then, and now this is clearly false [2, 3]. LeCun's post is based more in pride than reason; he spends most of the time talking about NeuFlow which at one point was a competitor to True North for funding. In the end, NeuFlow never became a chip, but…

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by partycoder

Fri, 14 Oct 2016 18:23:06 GMT

You can also try a binarized neural network. https://arxiv.org/abs/1602.02830

Paper and discussion history. A mention is not an endorsement.

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 — mentioned by cs702

Mon, 05 Sep 2016 16:32:17 GMT

As far as I understand this, these guys claim they can train convolutional and many other types of deep neural nets faster by pretraining each layer with a new unsupervised technique via which the layer sort of learns to compress its inputs (a local optimization problem), and then they fine tune the whole network end-to-end with supervised SGD and backpropagation as usual. They have not released code, so no one else has replicated this yet -- as far as I know. If the claim holds, the implication is that layers can quickly learn much of what they need to learn locally, that is, without…

Paper and discussion history. A mention is not an endorsement.