Faster Policy Learning with Continuous-Time Gradients · HackerPapers discussions

Faster Policy Learning with Continuous-Time Gradients — mentioned by ChrisRackauckas

Thu, 08 Sep 2022 20:45:57 GMT

The only ones that I know of are in https://arxiv.org/abs/2012.06684 (with Julia DifferentialEquations.jl and DiffTaichi), but those are more algorithmic. There, Julia does extremely well, but the conclusion that I would draw from it is more: use a programming language with robust and differentiable differential equation solvers rather than writing simple Euler loops by hand (as this article does).

Paper and discussion history. A mention is not an endorsement.

Faster Policy Learning with Continuous-Time Gradients — mentioned by klowrey

Wed, 01 Dec 2021 17:10:49 GMT

I manage the MuJoCo.jl wrapper, and having the source code wont (naively) help with AD. Internally, there's a number of iterative algorithms that you wouldn't want to automatically differentiate, and if analytical derivatives were apparent they would be in there (the MuJoCo creator was my advisor). As it stands, finite differencing is the best way to extract derivative/gradient information from the underlying physics model of MuJoCo, and is the suggested method. We had a paper that included MuJoCo (with custom defined adjoints) within Julia's DiffEq framework to learn continuous control…

Paper and discussion history. A mention is not an endorsement.

Faster Policy Learning with Continuous-Time Gradients — mentioned by ChrisRackauckas

Wed, 01 Dec 2021 11:08:21 GMT

>One of the domains where I hope Julia will excel is precisely as you comment on physics + ml (I check frequently if MuJoCo source code is available at last). Somebody already did a comparison of DifferentialEquations.jl's standard usage against MuJoCo and DiffTaichi and showed that it outperformed them by about 5x-10x. https://arxiv.org/abs/2012.06684 https://homes.cs.washington.edu/~thickstn/ctpg-project-page/... That's all showing the raw iteration count to show that it algorithmically is faster, but the time per iteration is also fast for many reasons showcased in the SciMLBenchmarks…

Paper and discussion history. A mention is not an endorsement.