Long short-term memory and learning-to-learn in networks of spiking neurons · HackerPapers discussions

Long short-term memory and learning-to-learn in networks of spiking neurons — mentioned by orbifold

Mon, 15 Feb 2021 22:36:14 GMT

You can actually calculate exact gradients for spiking neurons using the adjoint method: https://arxiv.org/abs/2009.08378 (I'm the second author). In my PhD thesis I show how this can be extended to larger problems and more complicated and biologically plausible neuron models. I agree with the gist of your post though: Retrofitting back propagation (or the adjoint method for that matter) is the wrong approach. One should rather use these methods to optimise biologically plausible learning rules. The group of Wolfgang Maass has done exciting work in that direction (e.g.…

Paper and discussion history. A mention is not an endorsement.

Long short-term memory and learning-to-learn in networks of spiking neurons — mentioned by trextrex

Sat, 02 Jun 2018 22:47:19 GMT

Many concerns in this paper, especially about deep learning being data hungry, limited capacity for transfer, and integrating prior knowledge are addressed by recent papers in meta-learning and learning-to-learn [1,2,3 and many many others] both in the supervised and reinforcement learning contexts. In the case of meta-reinforcement learning there has been recent work [5] which seems to indicate that this mechanism is very similar to how learning works in the brain. In fact, my group recently published a paper [4] about learning-to-learn in spiking networks (making it biologically more…

Paper and discussion history. A mention is not an endorsement.

Long short-term memory and learning-to-learn in networks of spiking neurons — mentioned by trextrex

Tue, 10 Apr 2018 19:09:16 GMT

Very cool. It's interesting how powerful the recurrent network becomes with the addition of the learned hebbian term. For context, even without the Hebbian term, recurrent networks can learn to learn to do quite interesting things (Hochreiter et al. 2001). Shameless plug -- our lab recently ported LSTMs to spiking networks without a significant loss in performance, and showed that learning to learn works quite well even with spiking networks (Bellec et al. 2018). So it seems like this method of learning to learn could provide a extremely biologically realistic and fundamental paradigm for…

Paper and discussion history. A mention is not an endorsement.