Reverse Differentiation via Predictive Coding · HackerPapers discussions

Reverse Differentiation via Predictive Coding — HN submission

Thu, 03 Jun 2021 20:53:31 GMT

Predictive Coding Can Do Exact Backpropagation on Any Neural Network

113 points; 9 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Reverse Differentiation via Predictive Coding — mentioned by pishpash

Sun, 02 May 2021 18:07:04 GMT

Also, most recent state of progress: Predictive Coding Can Do Exact Backpropagation on Any Neural Network (2021) https://arxiv.org/abs/2103.04689

Paper and discussion history. A mention is not an endorsement.

Reverse Differentiation via Predictive Coding — mentioned by autopoiesis

Mon, 05 Apr 2021 18:38:37 GMT

The paper (arxiv:2103.04689) linked by eutropia above has some empirical evidence on the ML side, showing that performance of predictive coding is not so far off backprop. And there is no shortage of suggestions for how neural circuits might work around the strict requirements of backprop-like algorithms. cs702's original comment above is excessively hyperbolic: the compositional structure of Bayesian inversion is well known and is known to coincide structurally with the backward/forward structure of automatic differentiation. And there have been many papers before this one showing how…

Paper and discussion history. A mention is not an endorsement.

Reverse Differentiation via Predictive Coding — mentioned by eutropia

Mon, 05 Apr 2021 17:55:04 GMT

Here's a more recent paper (March, 2021) which cites the above paper: https://arxiv.org/abs/2103.04689 "Predictive Coding Can Do Exact Backpropagation on Any Neural Network"

Paper and discussion history. A mention is not an endorsement.