On Exact Computation with an Infinitely Wide Neural Net — mentioned by mcilai

Sat, 25 Jan 2020 17:59:03 GMT

Deep learning is as close to being a "newtonian theory" of the brain as it gets -- deep learning abstracts away a lot of the complexity of neural systems (e.g., a simple artificial neuron vs a highly complex biological one) while maintaining a number of essential characteristics: massively parallel computation, error tolerance, graceful degradation, distributed representations, information is stored in slowly-changing synapses, and, most importantly: a simple, local, and powerful biologically-plausible-if-you-squint-hard-enough learning rule. The important question to ask is: is the deep…

Paper and discussion history. A mention is not an endorsement.

On Exact Computation with an Infinitely Wide Neural Net — mentioned by fgabriel

Thu, 28 Nov 2019 06:30:28 GMT

For trained wide neural networks, you can have a look at "Neural Tangent Kernel: Convergence and Generalization in Neural Networks" (https://arxiv.org/abs/1806.07572) where we explain the training of very wide ANN. This sparked a numerous amount of research and in the few last results about training wide ANN you can have a look at: https://arxiv.org/abs/1909.08156 and https://arxiv.org/abs/1904.11955

Paper and discussion history. A mention is not an endorsement.

On Exact Computation with an Infinitely Wide Neural Net · HackerPapers discussions

On Exact Computation with an Infinitely Wide Neural Net — mentioned by mcilai

On Exact Computation with an Infinitely Wide Neural Net — mentioned by fgabriel