HackerPapers

arXiv:2001.04385cs.LG

Universal Differential Equations for Scientific Machine Learning

Christopher Rackauckas, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, Alan Edelman

Abstract

In the context of science, the well-known adage "a picture is worth a thousand words" might well be "a model is worth a thousand datasets." In this manuscript we introduce the SciML software ecosystem as a tool for mixing the information of physical laws and scientific models with data-driven machine learning approaches. We describe a mathematical object, which we denote universal differential equations (UDEs), as the unifying framework connecting the ecosystem. We show how a wide variety of applications, from automatically discovering biological mechanisms to solving high-dimensional Hamilton-Jacobi-Bellman equations, can be phrased and efficiently handled through the UDE formalism and its tooling. We demonstrate the generality of the software tooling to handle stochasticity, delays, and implicit constraints. This funnels the wide variety of SciML applications into a core set of training mechanisms which are highly optimized, stabilized for stiff equations, and compatible with distributed parallelism and GPU accelerators.

arXiv comments
5 figures, 2 tables, 11 supplemental figures, 29 pages, 25 supplemental pages
Journal reference
DOI
Subjects
cs.LGMachine Learning math.DSDynamical Systems q-bio.QMQuantitative Methods stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 3 identifiable commenters in 8 comments across 6 threads, from Mar 2020 to May 2022. Found in 4 HN submissions from Jan 2020 to Aug 2020. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. ‘Machine Scientists’ Distill the Laws of Physics from Raw Data 1 comment

    • Yeah, Julia does great in this domain. DataDrivenDiffEq.jl (https://datadriven.sciml.ai/) is a comprehensive symbolic regression package out there, which pulls together all of what's mentioned in the article (SINDy and variants, EQSearch, SymbolicRegression.jl (which is the core…

      ChrisRackauckasView on HNParent context

  2. Differentiable Programming – A Simple Introduction 1 comment

    • It's a good question, which I answer in much more depth here: https://www.reddit.com/r/MachineLearning/comments/ryw53x/d_f... . Specifically that answer is about NNs vs fourier series, but it's the same point: polynomials, Fourier series, and NNs are all universal function…

      ChrisRackauckasView on HNParent context

  3. Differentiable Control Problems 1 comment

    • To respond to both the parent question, and this comment: indeed, this is black-box optimal control in essence. However, this method is just one small aspect of the SciML [0] ecosystem now. The article is a little outdated in that sense. Once obtaining your NN control parameter…

      LibbumView on HNParent context

  4. AI in physics: are we facing a scientific revolution? 3 comments

    • A lot of these methods here utilize the universal differential equation framework described here: https://arxiv.org/abs/2001.04385 . Specifically, the last example in this preprint describes how high dimensional parabolic PDEs can be solving using neural networks inside of a…

      ChrisRackauckasView on HNParent context

    • I'm not working on any specific problem yet, but for my master's thesis I'm hoping to do something related to the use of neural networks in numerical solutions to differential equations. Along the lines of this sort of stuff [2]. [2] https://arxiv.org/pdf/2001.04385.pdf

      cameronperotView on HNParent context

    • Yes, and our recent work https://arxiv.org/abs/2001.04385 gives a fairly general form for how to mix known scientific structural knowledge directly with machine learning. In fact, some of these PDE solvers are just instantiations of specific choices of universal differential…

      ChrisRackauckasView on HNParent context

  5. Ask HN: What’s a big trend we should all be following? 1 comment

    • Scientific machine learning. The ability to mix scientific knowledge with machine learning in order to allow it to extrapolate accurately from small amounts of data. As machine learning tries to move towards areas where large-scale data gathering is prohibitively expensive or…

      ChrisRackauckasView on HNParent context

  6. Ask HN: Who is hiring right now? 1 comment

    • The Julia Lab (MIT) | DevOps Engineer (+ More) | Onsite or Remote | Full Time | https://julia.mit.edu/ The Julia Lab is a lab in MIT's CSAIL which focuses on the development of software tooling for solving large-scale scientific problems. Named after it's original result, the…

      ChrisRackauckasView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.