Second-Order Neural ODE Optimizer — mentioned by ChrisRackauckas

Fri, 22 Oct 2021 10:37:14 GMT

Yes, that's mostly right (instead of PETSc put FATODE since PETSc TS Adjoint was only published in 2019, but it's based heavily on FATODE's techniques https://epubs.siam.org/doi/10.1137/130912335?mobileUi=0). And that's why the Julia tools were so ready for it: we already had adjoint sensitivity analysis (implemented for parameter estimation in systems pharmacology), so neural ODEs were a freebee. Similar to DEQs using the adjoint of a nonlinear solve: that means a DEQ is just a neural network inside of a Julia nonlinear solver and you call the gradients and you'll hit the right overloads…

Paper and discussion history. A mention is not an endorsement.

Second-Order Neural ODE Optimizer · HackerPapers discussions

Second-Order Neural ODE Optimizer — mentioned by ChrisRackauckas