Stochastic natural gradient descent draws posterior samples in function space — mentioned by duckworthd

Sun, 09 Dec 2018 02:59:36 GMT

My $0.02. I've been using JAX for a while now. A paper I'm an author on (https://arxiv.org/abs/1806.09597, a follow-up to https://news.ycombinator.com/item?id=18633215) resulted in an algorithm that required taking second-derivatives on a per-example basis. This is extremely difficult in TensorFlow, but with JAX it was a 2-liner. Even better, it's _super_ fast, thanks to XLA's compile-to-GPU and JAX's auto-batching mechanics. I highly recommend JAX to power users. It's nowhere near as feature-complete from a neural network sense as, say, PyTorch, but it is very good at what it does, and its…

Paper and discussion history. A mention is not an endorsement.

Stochastic natural gradient descent draws posterior samples in function space · HackerPapers discussions

Stochastic natural gradient descent draws posterior samples in function space — mentioned by duckworthd