A Stochastic Quasi-Newton Method for Large-Scale Optimization — mentioned by chestervonwinch

Sat, 04 Jan 2020 03:14:49 GMT

> You have a stochastic estimate since your loss is generally additive in the data. Just to be clear, additive loss doesn't imply stochastic gradient estimate. Rather, because the loss function is additive, then stochastic gradient estimates of the loss are now possible. But, this of course does not mean one has to use stochastic gradient estimates. It's just that it's easier to update and monitor progress this way, rather than computing the gradient term for every single example in the training set and then taking a descent step. The surprising thing is that stochastic gradient descent…

Paper and discussion history. A mention is not an endorsement.

A Stochastic Quasi-Newton Method for Large-Scale Optimization · HackerPapers discussions

A Stochastic Quasi-Newton Method for Large-Scale Optimization — mentioned by chestervonwinch