Nonlinear Approximation and (Deep) ReLU Networks — mentioned by joe_the_user

Sat, 23 Jan 2021 22:53:08 GMT

One thing to consider is the distinction between deep neural networks as mathematical objects and machine learning as currently practiced. Lately, there have been quite a few theories on neural networks as ideal nonlinear approximators [1]. Similarly, people have shown many ways that gradient descent can tend to reach a global maximum of regularized curve-closeness[2]. Which is to say, if your development cycle is: gather-data, train, test, deploy, we know this approximates the data almost ideally; you can't really do much better than a deep network. But we know in practice, when deployed…

Paper and discussion history. A mention is not an endorsement.

Nonlinear Approximation and (Deep) ReLU Networks · HackerPapers discussions

Nonlinear Approximation and (Deep) ReLU Networks — mentioned by joe_the_user