Towards Biologically Plausible Deep Learning · HackerPapers discussions

Towards Biologically Plausible Deep Learning — mentioned by kmmlng

Thu, 04 Jan 2024 14:53:18 GMT

What do you mean, the development of the "Perception"? Do you mean the Perceptron? In that case, Backprop was invented way later than the Perceptron (see https://people.idsia.ch/~juergen/who-invented-backpropagatio...). I don't see any information in your linked Wikipedia article that supports a bio-inspired origin. In fact, researchers have been wondering whether an equivalent to Backprop might be found in biological brains, but Backprop is widely believed to be biologically implausible (see e.g. https://arxiv.org/pdf/1502.04156.pdf…

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — mentioned by throwawaywego

Tue, 23 Jul 2019 00:26:41 GMT

https://arxiv.org/abs/1502.04156 > Towards Biologically Plausible Deep Learning > Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology. We explore more biologically plausible versions of deep representation learning, focusing here mostly on unsupervised learning but developing a learning mechanism that could account for supervised, unsupervised and reinforcement learning. The starting point is that the basic learning rule believed to govern synaptic weight updates (Spike-Timing-Dependent Plasticity) arises out of a simple update…

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — mentioned by ben_w

Tue, 11 Dec 2018 16:25:07 GMT

> We're not deep learning systems. So reasoning by analogy there isn't correct. That’s why I’m asking which definition you’re using. It’s broad enough that some models are biologically plausible even though there is criticism that many others are not: https://arxiv.org/abs/1502.04156 > Deep learning systems, are examples of supervised learning. Once trained, they cannot adapt to new input. You have to retrain to accommodate new features that you want to capture. Supervised, unsupervised, and reinforcement; and there is work on both incremental learning (https://arxiv.org/abs/1712.02719) and…

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — mentioned by iandanforth

Mon, 23 Oct 2017 18:04:06 GMT

I might start with this paper from Bengio et al. and then read more recent papers which cite it. The problem hasn't been solved though so don't be surprised if you're not fully convinced by the arguments :) https://arxiv.org/pdf/1502.04156.pdf

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — mentioned by trott

Sat, 02 Sep 2017 19:41:05 GMT

> Every time a major advance is made in AI the computational neuroscientist say: "why didn't you talk to us 15 years ago? We could have told you that!". I'm not sure which advances you are talking about, but the modern successes of Deep Learning are primarily due to backpropagation and convolution. Neither of the two is considered to be biologically plausible. Some people are actively trying to come up with alternatives to backpropagation that would be biologically plausible though. See for example: Bengio et al., "Towards Biologically Plausible Deep Learning" https://arxiv.org/abs/1502.04156

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — mentioned by mattkrause

Wed, 21 Dec 2016 02:45:38 GMT

(cont) I probably should have lead with this, but there's been a lot of interest in backprop-like algorithms in the brain * Geoff Hinton has a talk (and slide deck) about how back-propagation might be implemented in the brain. (Slide deck: https://www.cs.toronto.edu/~hinton/backpropincortex2014.pdf Video: http://sms.cam.ac.uk/media/2017973?format=mpeg4&quality=720p ) * As always, the French part of Canada has its own, slightly different version of things, care of Yoshua Bengio (slide deck from NIPS 2015: https://www.iro.umontreal.ca/~bengioy/talks/NIPS2015_NeuralS... preprint…

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — mentioned by tlb

Tue, 20 Dec 2016 06:38:27 GMT

There is some work in this direction. See Bengio's Biologically Plausible Deep Learning (https://arxiv.org/pdf/1502.04156.pdf) and its references for a starting point. So far, they don't achieve SOTA on anything.

Paper and discussion history. A mention is not an endorsement.

Towards Biologically Plausible Deep Learning — HN submission

Sat, 05 Sep 2015 00:27:11 GMT

Towards Biologically Plausible Deep Learning

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.