HackerPapers

arXiv:1802.01528cs.LG

The Matrix Calculus You Need For Deep Learning

Terence Parr, Jeremy Howard

Abstract

This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. We assume no math knowledge beyond what you learned in calculus 1, and provide links to help you refresh the necessary math where needed. Note that you do not need to understand this material before you start learning to train and use deep learning in practice; rather, this material is for those who are already familiar with the basics of neural networks, and wish to deepen their understanding of the underlying math. Don't worry if you get stuck at some point along the way---just go back and reread the previous section, and try writing down and working through some examples. And if you're still stuck, we're happy to answer your questions in the Theory category at forums.fast.ai. Note: There is a reference section at the end of the paper summarizing all the key matrix calculus rules and terminology discussed here. See related articles at http://explained.ai

arXiv comments
PDF version of mobile/web friendly version http://explained.ai/matrix-calculus/index.html
Journal reference
DOI
Subjects
cs.LGMachine Learning stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 7 identifiable commenters in 7 comments across 7 threads, from Aug 2018 to Mar 2025. Found in 6 HN submissions from Feb 2018 to Jan 2025. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments7 threads

  1. Matrix Calculus (For Machine Learning and Beyond) 1 comment
  2. Ask HN: Calculus Refresher for AI/ML? 1 comment

    • The Matrix Calculus You Need For Deep Learning https://arxiv.org/abs/1802.01528 From the intro: >This article walks through the derivation of some important rules for computing partial derivatives with respect to vectors, particularly those useful for training neural networks.…

      woolionView on HNParent context

  3. Calculus on Computational Graphs: Backpropagation (2015) 1 comment

    • If you want more on this, check out this paper by Terence Parr and Jeremy Howard: https://arxiv.org/abs/1802.01528 It is rather accessible.

      sva_View on HNParent context

  4. The matrix calculus you need for deep learning (2018) 1 comment

    • Please change the link to the original source: https://arxiv.org/abs/1802.01528 --- EDIT: It turns out explained.ai is the personal website of one of the authors, so there's no need to change the link. See comment below.

      cs702View on HNParent context

  5. Matrix Calculus: Calculate derivatives of matrices 1 comment
  6. Ask HN: As a professional, does it worth it to learn the math behind NN? 1 comment
  7. Learning Math for Machine Learning 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.