No More Pesky Learning Rates — mentioned by ogrisel

Mon, 25 Jun 2012 01:05:08 GMT

One unmentioned caveat of SGD is how to configure the learning rate schedule. scikit-learn is using Bottou's tricks that seem to work reasonably well in practice but it might even be better to implement the online estimate of optimal learning rate schedule from this NIPS 2012 pre-print: http://arxiv.org/abs/1206.1106 (No More Pesky Learning Rates).

Paper and discussion history. A mention is not an endorsement.

No More Pesky Learning Rates · HackerPapers discussions

No More Pesky Learning Rates — mentioned by ogrisel