Universal Differential Equations for Scientific Machine Learning · HackerPapers discussions

Universal Differential Equations for Scientific Machine Learning — mentioned by ChrisRackauckas

Wed, 11 May 2022 13:28:11 GMT

Yeah, Julia does great in this domain. DataDrivenDiffEq.jl (https://datadriven.sciml.ai/) is a comprehensive symbolic regression package out there, which pulls together all of what's mentioned in the article (SINDy and variants, EQSearch, SymbolicRegression.jl (which is the core of PySR), OccamNet, etc.) all into a single API to allow comparisons between each of the methods. The Julia automated discovery of physical equations work showcases these symbolic regression techniques quite heavily, for example in the State of SciML talk (https://www.youtube.com/watch?v=eSeY4K4bITI) we discuss how…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by ChrisRackauckas

Thu, 14 Apr 2022 10:41:46 GMT

It's a good question, which I answer in much more depth here: https://www.reddit.com/r/MachineLearning/comments/ryw53x/d_f... . Specifically that answer is about NNs vs fourier series, but it's the same point: polynomials, Fourier series, and NNs are all universal function approximators, so why use a NN? If you write out what happens though, you can easily see the curse of dimensionality in action and see why a 100-dimensional function shouldn't be approximated by tensor products of polynomials. See the rest there. So it's not really about NNs vs polynomials/fourier/Chebyshev, etc., but…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by Libbum

Thu, 27 Aug 2020 08:49:15 GMT

To respond to both the parent question, and this comment: indeed, this is black-box optimal control in essence. However, this method is just one small aspect of the SciML [0] ecosystem now. The article is a little outdated in that sense. Once obtaining your NN control parameter, it's now possible to use Sparse Identification of Nonlinear Dynamics (SINDy) on that parameter to recover equations of motion governing it [1]. The real promise of these methods is to use the universal approximator power of NNs to get around the 'curse of dimensionality' & uncover presently unknown representations of…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — HN submission

Thu, 06 Aug 2020 11:57:40 GMT

Universal Differential Equations for Scientific Machine Learning

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by ChrisRackauckas

Tue, 21 Jul 2020 20:43:52 GMT

Yes, and our recent work https://arxiv.org/abs/2001.04385 gives a fairly general form for how to mix known scientific structural knowledge directly with machine learning. In fact, some of these PDE solvers are just instantiations of specific choices of universal differential equations. I agree that in many cases the "fully uninformed" physics-informed neural network won't work well, but we need to fully optimize a library with all of the training techniques possible in order to prove that, which is what we plan to do. In the end, I think PINNs will be most applicable to (1) non-local PDEs…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by cameronperot

Tue, 21 Jul 2020 16:12:56 GMT

I'm not working on any specific problem yet, but for my master's thesis I'm hoping to do something related to the use of neural networks in numerical solutions to differential equations. Along the lines of this sort of stuff [2]. [2] https://arxiv.org/pdf/2001.04385.pdf

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by ChrisRackauckas

Tue, 21 Jul 2020 16:12:03 GMT

A lot of these methods here utilize the universal differential equation framework described here: https://arxiv.org/abs/2001.04385 . Specifically, the last example in this preprint describes how high dimensional parabolic PDEs can be solving using neural networks inside of a specific SDE (derivation in the supplemental). Discrete physics-informed neural networks also are a subset of this methodology. The other subset of methods, continuous physics-informed neural networks, are described in https://www.sciencedirect.com/science/article/pii/S002199911... . For a very basic introduction, I wrote…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by ChrisRackauckas

Sat, 20 Jun 2020 04:06:51 GMT

Scientific machine learning. The ability to mix scientific knowledge with machine learning in order to allow it to extrapolate accurately from small amounts of data. As machine learning tries to move towards areas where large-scale data gathering is prohibitively expensive or even unethical (like in clinical trials: you can't have a million failures to train a neural network!), mixing scientific models with machine learning will be required to bring AI into these domains in a way that's beyond hype. Physics-informed neural networks [1], differentiable programming [2], universal differential…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — HN submission

Sun, 17 May 2020 16:05:24 GMT

Universal Differential Equations for Scientific Machine Learning

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — HN submission

Mon, 04 May 2020 17:39:19 GMT

Universal Differential Equations for ScientificMachine Learning

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — mentioned by ChrisRackauckas

Mon, 23 Mar 2020 16:47:19 GMT

The Julia Lab (MIT) | DevOps Engineer (+ More) | Onsite or Remote | Full Time | https://julia.mit.edu/ The Julia Lab is a lab in MIT's CSAIL which focuses on the development of software tooling for solving large-scale scientific problems. Named after it's original result, the Julia programming language, the Julia Lab's research ranges from scientific machine learning (SciML) [1] and differentiable programming [2] to large-scale heterogeneous parallel computing [3]. The lab is currently growing in response to new directions and collaborations, and we are looking for both researchers and devops…

Paper and discussion history. A mention is not an endorsement.

Universal Differential Equations for Scientific Machine Learning — HN submission

Tue, 14 Jan 2020 15:37:49 GMT

Universal Differential Equations for Scientific Machine Learning

6 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.