Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks · HackerPapers discussions

Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks — mentioned by orbifold

Thu, 07 Dec 2023 15:40:04 GMT

That you can't backpropagate is a common misconception. There is recent work (I am one of the co-authors), that derives a precise analog of the backpropagation algorithm in spiking neural networks (https://arxiv.org/abs/2009.08378). It computes exact gradients and requires only communication at spike times during the backward pass. The reason this works is because "x is not differentiable" lacks a statement "with respect to y". It turns out that since gradient computation is local, a gradient is defined almost everywhere. The gradient computation is only ill-defined at places where a spike…

Paper and discussion history. A mention is not an endorsement.

Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks — HN submission

Wed, 02 Jun 2021 04:17:14 GMT

Event-based backpropagation for exact gradients in spiking neural networks

119 points; 37 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks — mentioned by orbifold

Mon, 15 Feb 2021 22:36:14 GMT

You can actually calculate exact gradients for spiking neurons using the adjoint method: https://arxiv.org/abs/2009.08378 (I'm the second author). In my PhD thesis I show how this can be extended to larger problems and more complicated and biologically plausible neuron models. I agree with the gist of your post though: Retrofitting back propagation (or the adjoint method for that matter) is the wrong approach. One should rather use these methods to optimise biologically plausible learning rules. The group of Wolfgang Maass has done exciting work in that direction (e.g.…

Paper and discussion history. A mention is not an endorsement.

Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks — HN submission

Fri, 18 Sep 2020 08:13:29 GMT

EventProp: Backpropagation for Exact Gradients in Spiking Neural Networks

8 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.