Training Neural Networks Without Gradients: A Scalable ADMM Approach — mentioned by thesz

Thu, 18 Feb 2021 00:54:13 GMT

I beg to disagree. [1] provides one with a whole-data-set training method (ADMM, one of such methods). Page 8 contains figure 2(b) - accuracy of training after specified amount of time. Note that ADMM start where stochastic gradient stops. [1] https://arxiv.org/pdf/1605.02026.pdf At [2] I tried to apply logistic regression trained using reweighted least squares algorithm on the same Higgs boson data set. I've got the same accuracy (64%) as mentioned in the ADMM paper with much less number of coefficients - basically, just the size of input vector + 1 instead of 300 such rows of coefficients…

Paper and discussion history. A mention is not an endorsement.

Training Neural Networks Without Gradients: A Scalable ADMM Approach — mentioned by thesz

Tue, 29 May 2018 09:24:27 GMT

>This stochasticity seems important to achieving good results. No, it is not and may be counter resultive, so to say. https://arxiv.org/pdf/1605.02026.pdf - page 8, figure 2(b). SGD optimized neural networks stops learning at the accuracy at which whole-dataset methods start! Also please note that the figure I pointed to is about high energy particles analysis. SGD trained NN cannot even distinguish particles with good precision, let alone discover physics.

Paper and discussion history. A mention is not an endorsement.

Training Neural Networks Without Gradients: A Scalable ADMM Approach · HackerPapers discussions

Training Neural Networks Without Gradients: A Scalable ADMM Approach — mentioned by thesz

Training Neural Networks Without Gradients: A Scalable ADMM Approach — mentioned by thesz