A Convergence Theory for Deep Learning via Over-Parameterization · HackerPapers discussions

A Convergence Theory for Deep Learning via Over-Parameterization — mentioned by kvathupo

Thu, 19 Jan 2023 21:22:27 GMT

I think the battle to promote the use of quantum computers is rendered even more difficult by the fact that their raison d'être of "being faster than classical computers" requires one to further develop the existing field of complexity analysis for classical computers. Indeed, it's difficult to fairly compare the gate/oracle-based complexity of quantum computers to the iterations/multiplications/etc-based complexity of classical computers, especially if we don't know their complexity in the first place [1]! [1] - Recently, there has been interest in the complexity analysis of deep learning…

Paper and discussion history. A mention is not an endorsement.

A Convergence Theory for Deep Learning via Over-Parameterization — mentioned by gogut

Wed, 14 Nov 2018 21:52:12 GMT

This paper appears in November, but in fact, Allen-Zhu (MSR, http://people.csail.mit.edu/zeyuan/ ) already posted his result in Oct. This is their first paper in Oct:https://arxiv.org/pdf/1810.12065.pdf, this is their second paper in Nov https://arxiv.org/pdf/1811.03962.pdf . In MSR Oct paper, they proved how to train RNN (which is even harder than DNN). In their Nov paper, they proved how to train DNN. Compared to their Oct one, the Nov one is actually much easier. The reason is, in RNN, every layer has the same weight matrix, but in DNN every layer could have different weight matrices.…

Paper and discussion history. A mention is not an endorsement.