Deep Neural Networks as Gaussian Processes · HackerPapers discussions

Deep Neural Networks as Gaussian Processes — mentioned by mildmay_rider

Sun, 25 Feb 2024 09:40:51 GMT

You may enjoy some works on the connections between gaussian processes, neural networks, and linear(in rkhs) models. https://papers.nips.cc/paper_files/paper/2019/hash/39d929972... https://papers.nips.cc/paper_files/paper/1996/hash/ae5e3ce40... https://arxiv.org/abs/1711.00165

Paper and discussion history. A mention is not an endorsement.

Deep Neural Networks as Gaussian Processes — mentioned by nickpsecurity

Sun, 25 Feb 2024 02:59:41 GMT

Thanks for the submission. I keep an eye out for connections to potentially cheaper and simpler methods. The new comparison reminds me of this old paper which compared them to Gaussian networks: https://arxiv.org/abs/1711.00165 For causality, I'm also keeping an eye out for ways to tie DNN research back into decision trees, esp probabilistic and nested. Or fuzzy logic. Here's an example I saw moving probabilistic methods into decision trees: http://www.gatsby.ucl.ac.uk/~balaji/balaji-phd-thesis.pdf Many of these sub-fields develop separately in their own ways. Gotta wonder what recent…

Paper and discussion history. A mention is not an endorsement.

Deep Neural Networks as Gaussian Processes — mentioned by PartiallyTyped

Fri, 23 Jun 2023 20:43:02 GMT

> How well does a classic deep net architecture like AlexNet or VGG19 classify on a standard dataset such as CIFAR-10 when its “width”— namely, number of channels in convolutional layers, and number of nodes in fully-connected internal layers — is allowed to increase to infinity? Such questions have come to the forefront in the quest to theoretically understand deep learning and its mysteries about optimization and generalization. They also connect deep learning to notions such as Gaussian processes and kernels. A recent paper [Jacot et al., 2018] introduced the Neural Tangent Kernel (NTK)…

Paper and discussion history. A mention is not an endorsement.

Deep Neural Networks as Gaussian Processes — mentioned by medlyyy

Fri, 05 Mar 2021 20:34:58 GMT

I believe it's this: https://arxiv.org/abs/1711.00165 And then the NTK paper: https://arxiv.org/abs/1806.07572 There is also this which seems to have been produced in parallel, closely related to the first but not exactly the same: https://arxiv.org/abs/1804.11271

Paper and discussion history. A mention is not an endorsement.

Deep Neural Networks as Gaussian Processes — mentioned by yorwba

Sat, 04 Jan 2020 11:12:08 GMT

You can do basic math to a multivariate normal distribution to approximate other distributions. In classic feedforward networks, that's achieved by using nonlinear activation functions. Very wide networks turn into Gaussian processes in the limit [0]. There are also approaches explicitly embracing the "basic math on multivariate normal distributions" framework using normalizing flows [1]. [0] https://arxiv.org/abs/1711.00165 [1] https://arxiv.org/abs/1505.05770

Paper and discussion history. A mention is not an endorsement.

Deep Neural Networks as Gaussian Processes — mentioned by vladislav

Mon, 23 Jul 2018 00:27:46 GMT

There are some interesting connections between GPs and neural networks. Deep neural networks with random weights concentrate about gaussian processes, so the two may not be so far apart in practice. https://arxiv.org/pdf/1711.00165.pdf

Paper and discussion history. A mention is not an endorsement.

Deep Neural Networks as Gaussian Processes — mentioned by nabla9

Fri, 26 Jan 2018 11:58:58 GMT

How exactly did theoretical equivalence result kill SOM's as field of study? There is similar connection between deep fully connected neural networks and Gaussian process. Deep Neural Networks as Gaussian Processes https://arxiv.org/abs/1711.00165

Paper and discussion history. A mention is not an endorsement.