Saddle-free Hessian-free Optimization — mentioned by jph00

Sat, 06 May 2017 22:04:18 GMT

It's not so much that there's just one or two specific results, but rather that there's just far more researchers working on this now, and quite a few are working on more theoretical stuff. Sometimes, that theory results in really practical outcomes - a good example would be https://arxiv.org/abs/1701.07875 . Or another by the same researcher: https://arxiv.org/abs/1506.00059 . We've seen, in the last year or two, interesting results in nearly every area of theoretical research of deep learning, including generalization, optimization, generative modelling, bayesian models, and network…

Paper and discussion history. A mention is not an endorsement.

Saddle-free Hessian-free Optimization · HackerPapers discussions

Saddle-free Hessian-free Optimization — mentioned by jph00