Vprop: Variational Inference using RMSprop — mentioned by imurray

Tue, 19 Mar 2019 13:58:03 GMT

I wouldn't recommend this to someone wanting to use the currently most practical tools. But I'm glad some people want to know about this stuff. Sometimes the natural gradient turns out to be easier to compute than the gradient, and then it has made it in to applications. Variational Inference: A Review for Statisticians (Blei et al., 2016) https://arxiv.org/abs/1601.00670 In this case, the gradient (51) has the Fisher Information in it and the natural gradient gets rid of that hard-to-compute term. This stuff has been used in a bunch of applications (section 5.1, p23). You could argue that…

Paper and discussion history. A mention is not an endorsement.

Vprop: Variational Inference using RMSprop · HackerPapers discussions

Vprop: Variational Inference using RMSprop — mentioned by imurray