Neural Ordinary Differential Equations · HackerPapers discussions

Neural Ordinary Differential Equations — mentioned by rfonseca

Thu, 07 Mar 2024 12:49:30 GMT

This paper [1] models some simple (r) NN as ODEs, and uses ODE tools to train and for inference. It’s a start. [1] https://arxiv.org/abs/1806.07366

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by Scene_Cast2

Fri, 01 Mar 2024 15:23:32 GMT

People have mentioned the discrete - continuous tradeoff. One way to bridge that gap would be to use https://arxiv.org/abs/1806.07366 - they draw an equivalence between vanilla (FC layer) neural nets of constant width with differential equations, and then use a differential equation solver to "train" a "neural net" (from what I remember - it's been years since that paper...). Another approach might be to take an information theoretic view with the infinite-width finite-entropy nets.

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by SemanticStrengh

Sat, 30 Apr 2022 07:07:35 GMT

Related https://arxiv.org/abs/1806.07366

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by rubatuga

Tue, 30 Mar 2021 09:36:39 GMT

Yeah. This kind of mirrors my thought that our visual cortex is excellent at split-second image recognition, for things like identifying predators, dangers, etc, and this lack of "thinking" is why our current generation of neural networks has matched the performance of humans on image recognition tasks. I agree that there is a "time" component necessary for improved thinking or intelligence, and since our current neural networks don't have these dynamics, they are lacking. Extremely deep neural networks, especially with residual connections, IMO are better approximations of "thinking". Models…

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by angel_j

Thu, 26 Sep 2019 21:35:43 GMT

Neural ODEs? "Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. " - https://arxiv.org/abs/1806.07366

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by damip

Tue, 26 Mar 2019 00:13:02 GMT

This work focuses on steady-state computing, but it could also be interesting to use transient physical behaviors to process time-varying signals Maybe by modelling dynamical systems as "neural nets" as in: https://arxiv.org/abs/1806.07366 and https://arxiv.org/abs/1808.08412 Or by using complicated physical systems we don't even understand to build Echo State Networks: http://www.scholarpedia.org/article/Echo_state_network

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by cs702

Fri, 18 Jan 2019 17:53:05 GMT

Nice work. I can think of only two things that are missing: * Normalizing flows - e.g., https://arxiv.org/abs/1605.08803 , https://arxiv.org/abs/1807.03039 , among many others * ODEnets and continous normalizing flows - https://arxiv.org/abs/1806.07366

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by joshuamorton

Mon, 14 Jan 2019 17:47:41 GMT

>We've got a winner. Tensors "do" something; they're functions. And a neural network is just a stack of linear transforms (the "tensors") and nonlinear transforms (the activations). So in a lot of cases, these "tensors" do do something. Granted the abstraction for storing the active tensors is also a great abstraction for storing high-dimensional data (like a 128x128x4x60 2-second video). >neural networks as ODEs You're a few months late :P [0] [0]: https://arxiv.org/abs/1806.07366

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — HN submission

Thu, 13 Dec 2018 23:09:12 GMT

Neural Ordinary Differential Equations

240 points; 60 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Ordinary Differential Equations — HN submission

Thu, 13 Dec 2018 05:25:57 GMT

Neural Ordinary Differential Equations

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Neural Ordinary Differential Equations — mentioned by nmstoker

Wed, 12 Dec 2018 17:42:07 GMT

The paper the article is based on is here: https://arxiv.org/abs/1806.07366 And there was mention on HN about a month ago of a PyTorch implementation for ODEs (the name for this new NN design) here: https://github.com/rtqichen/torchdiffeq/blob/master/README.m...

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by ArtWomb

Tue, 04 Dec 2018 01:28:47 GMT

Neural ODE solver just announced as a Best Paper at NeurIPS 2018 conference. What's interesting is that not only do preliminary results yield significantly less error in prediction vs RNNs. Differentiable models allow for extrapolations well beyond range of observable test data set. Neural Ordinary Differential Equations https://arxiv.org/pdf/1806.07366.pdf

Paper and discussion history. A mention is not an endorsement.

Neural Ordinary Differential Equations — mentioned by MarkMMullin

Sun, 24 Jun 2018 15:36:01 GMT

I've long wondered about this - I see and understand the comments by people pointing out this paper may a bit shallow - in the same sort of vein, I have this on my desk to grind thru - interested if anyone else has looked at it -- https://arxiv.org/abs/1806.07366

Paper and discussion history. A mention is not an endorsement.