https://hackerpaper.kaipfister.de/p/1611.01144/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=14771512urn:hackerpapers:mention:1611.01144:14771512Fri, 14 Jul 2017 17:27:56 GMT<p>No, stochastic gradient descent (SGD) needs a differentiable loss function (differentiable with respect to the trainable parameters). The stochastic bit is that you want to optimize the loss for the whole dataset but what you actually do is to do each gradient descent step on a small mini-batch of the dataset. There might be extensions where you have some non-differentiable part where you just assume some gradient in back-propagation. Like Gumble-Softmax (https://arxiv.org/abs/1611.01144). Or similar is the reparametrization trick in VAEs (https://arxiv.org/abs/1606.05908). But that are…</p><p><a href="https://hackerpaper.kaipfister.de/p/1611.01144/">Paper and discussion history</a>. A mention is not an endorsement.</p>stat.ML