On the Convergence Rate of Training Recurrent Neural Networks — mentioned by gogut

Wed, 14 Nov 2018 21:52:12 GMT

This paper appears in November, but in fact, Allen-Zhu (MSR, http://people.csail.mit.edu/zeyuan/ ) already posted his result in Oct. This is their first paper in Oct:https://arxiv.org/pdf/1810.12065.pdf, this is their second paper in Nov https://arxiv.org/pdf/1811.03962.pdf . In MSR Oct paper, they proved how to train RNN (which is even harder than DNN). In their Nov paper, they proved how to train DNN. Compared to their Oct one, the Nov one is actually much easier. The reason is, in RNN, every layer has the same weight matrix, but in DNN every layer could have different weight matrices.…

Paper and discussion history. A mention is not an endorsement.

On the Convergence Rate of Training Recurrent Neural Networks · HackerPapers discussions

On the Convergence Rate of Training Recurrent Neural Networks — mentioned by gogut