Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator · HackerPapers discussions

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — mentioned by johnnycerberus

Sat, 13 Feb 2021 15:17:06 GMT

I really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol. There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1] DeepLearning.scala that does compile-time reverse AD. [2] Compute.scala for operator fusion at runtime (JITifying) [3] Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time. Moreover, Kotlin also has a library that does type-safe AD. [5] nVidia and Oracle want…

Paper and discussion history. A mention is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — HN submission

Sun, 16 Feb 2020 19:24:07 GMT

Demystifying Differentiable Programming (2018)

110 points; 42 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — HN submission

Sun, 18 Aug 2019 15:34:52 GMT

Demystifying Differentiable Programming: Shift/Reset, Penultimate Backpropagator

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — mentioned by JadeNB

Wed, 13 Feb 2019 02:06:21 GMT

When possible, one should link to the abstract page, if for no other reason than that it makes version tracking easier: Want, Wu, Essertel, Decker, and Rompf - Demystifying differentiable programming: shift/reset the penultimate backpropagator (https://arxiv.org/abs/1803.10228), a title which is entirely too cute for its own good.

Paper and discussion history. A mention is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — mentioned by ricksharp

Wed, 13 Feb 2019 02:03:37 GMT

I found this paper that helps answer the question: https://arxiv.org/pdf/1803.10228.pdf

Paper and discussion history. A mention is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — mentioned by YorkshireSeason

Tue, 20 Nov 2018 09:28:58 GMT

Also not mentioned in the Baydin et al paper (which has been knocking about for a while: it was submitted in 2017, so not surprising): * F. Wang, X. Wu, G. Essertel, J. Decker, T. Rompf, Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator, see [1], and [4] for a presentation based on this paper. * S. Laue, M. Mitterreiter, J. Giesen, Computing Higher Order Derivatives of Matrix and Tensor Expressions, see [2], discussed in [3]. Quite a bit of exciting work in better understanding backpropagation going on right now. [1] https://arxiv.org/abs/1803.10228 [2]…

Paper and discussion history. A mention is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — HN submission

Sun, 06 May 2018 20:24:53 GMT

Demystifying Differentiable Programming (2018)

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator — HN submission

Tue, 03 Apr 2018 17:30:13 GMT

Connections Between Automatic Differentiation and Delimited Continuations

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.