HackerPapers

arXiv:1904.02990cs.SC

On the Equivalence of Automatic and Symbolic Differentiation

Soeren Laue

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

We show that reverse mode automatic differentiation and symbolic differentiation are equivalent in the sense that they both perform the same operations when computing derivatives. This is in stark contrast to the common claim that they are substantially different. The difference is often illustrated by claiming that symbolic differentiation suffers from "expression swell" whereas automatic differentiation does not. Here, we show that this statement is not true. "Expression swell" refers to the phenomenon of a much larger representation of the derivative as opposed to the representation of the original function.

arXiv comments
Journal reference
DOI
Subjects
cs.SCSymbolic Computation cs.LGMachine Learning

On Hacker News

Linked by 1 identifiable commenter in 2 comments across 2 threads, from Apr 2023 to Aug 2025.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Does the Bitter Lesson Have Limits? 1 comment

    • You could argue that since automatic differentiation and symbolic differentiation are equivalent, [1] symbolic AI did succeed by becoming massively parallelizable, we just needed to scale up the data and hardware in kind. [1]: https://arxiv.org/pdf/1904.02990

      bmc7505View on HNParent context

  2. Matrix Calculus: Calculate derivatives of matrices 1 comment

    • Sören Laue, creator and first author of this work [1] gave a nice talk [2,3] about it at our reading group not long ago. He also has written about how automatic and symbolic differentiation are essentially equivalent. [4] [1]…

      bmc7505View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.