An exact mapping between the Variational Renormalization Group and Deep Learning · HackerPapers discussions

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by riemannzeta

Tue, 20 May 2025 22:47:40 GMT

Thanks for this additional link, which really underscores for me at least how you're right about patterns in circuits being a better abstraction layer for capturing interesting patterns than topological manifolds. I wasn't familiar with the term "equivariance" but I "woke up" to this sort of approach to understanding deep neural networks when I read this paper, which shows how restricted boltzman machines have an exact mapping to the renormalization group approach used to study phase transitions in condensed matter and high energy physics: https://arxiv.org/abs/1410.3831 At high enough…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by riemannzeta

Fri, 07 Jun 2024 14:19:58 GMT

Love this curious and open-minded exploration of how this stuff works. The pyramid strategy loosely tracks with renormalization group theory, which has been formally studied for years as a method of interpreting machine learning models: https://arxiv.org/abs/1410.3831 I love the convergence we're seeing in the use of models from different fields to understand machine learning, fundamental physics, and human consciousness. What a time to be alive.

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by mlevental

Sat, 27 Jul 2019 19:33:48 GMT

it's quite plausible that at the fringe/edge there is research being done that involves differential geometry - hell there's this paper https://arxiv.org/pdf/1410.3831v1.pdf that's occupies very rarefied air, but that's not what the person to whom i'm responding to claims (that somewhere someone there's a relationship): >DG is the foundation of all modern AI/ML/DL which is just blather spoken in a tone of confidence in order to sound very smart.

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by cs702

Fri, 25 Aug 2017 23:23:23 GMT

Thank you for posting this. Similar thoughts have been percolating in my stream-of-consciousness for a while, especially after coming across an earlier version of Tegmark & Lin's paper some months ago. My take is that deep neural nets work so well in practice because not all distributions of natural data are equally likely (and therefore, the no-free-lunch theorem(s), which assumes all distributions are equally likely, doesn't hold in the real world!); and that, in turn, is because the distribution of natural data is a consequence of the laws of Physics / symmetries of the universe in which…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by epberry

Mon, 05 Dec 2016 19:28:57 GMT

Part of me says that you are correct on all of these points. However I think you went too far on 3. Experimentation does not always follow theory in science. I would argue that many of the great discoveries of physics in the 20th century followed directly from startling results of experiments conducted at the end of the 19th century (the photoelectric effect for example). I agree that there seems to be an awful lot of experimenting going on and that hardware and large datasets have helped tremendously but there are also many researchers poking and prodding at deep learning theory [1, 2]. So…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — HN submission

Wed, 30 Nov 2016 01:55:18 GMT

Exact mapping between Variational Renormalization Group and Deep Learning (2014)

97 points; 13 discussion replies on HN. Paper and source history. A submission is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by ssivark

Mon, 28 Nov 2016 07:36:03 GMT

I am working on a project (using Julia) to "learn" an efficient representation of scale invariant many-body quantum states from a microscopic Hamiltonian. The particular structure being learned is supposed to be highly efficient both in representational complexity and computational complexity for predicting physical observables. (Work in progress at https://github.com/sivark/jumera/ ) Also reading/thinking about ideas at the heart of theoretical physics and machine learning. In particular, I've spent the last several months in trying to understand the following two papers in increasing levels…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by adityab

Sat, 10 Sep 2016 10:00:02 GMT

You should watch Max Tegmark's talk "Connections between Physics and Deep Learning" if this interests you [1]. Additionally, a paper that has everyone excited about deep connections between the mathematical analysis of physical systems and the hierarchical feature learning paradigm speaks of the connection in terms of the Renormalization Group [2]. Regardless of it's practical utility, the philosophical implications do tickle the intellect. On a dreamy note, I wonder if it would be possible to draw Category-theoretic parallels between some physical theories and statistical learning theory.…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — HN submission

Fri, 31 Jul 2015 14:50:56 GMT

An Exact Mapping Between the Variational Renormalization Group and Deep Learning

30 points; 7 discussion replies on HN. Paper and source history. A submission is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by T-A

Tue, 21 Jul 2015 14:55:42 GMT

To establish in a semi-rigorous way the (btw obvious) relationship between coarsening in physics and in stacked neural networks, as was done long after the latter became a thing [1][2], is at the same level as observing the analogy between Black-Scholes and diffusion. It's cute, but it does not turn the subject into physics. It should go without saying that you can not, by definition, resolve fundamental questions about mathematical physics by experiment. The APS took a stand on climate change years ago [3]. It was clear enough to make Harold Lewis [4] and Ivar Glaever [5] resign in protest.…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — HN submission

Thu, 05 Feb 2015 20:06:09 GMT

An Exact Mapping Between the Variational Renormalization Group and Deep Learning

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — HN submission

Mon, 08 Dec 2014 01:44:51 GMT

Deep Learning and Renormalization Group Thery

10 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by thisisdave

Thu, 04 Dec 2014 23:37:24 GMT

This could be an important result, since there's a lot of theory and expertise regarding renormalization group methods, but I think it's too early to tell. The write-up is a bit misleading, though: the model in their preprint[1] is about stacks of restricted Boltzmann machines (RBMs), and are very different from the other examples of deep learning mentioned. The Google Cat Detector model, for example, didn't describe a probability distribution over images, which is the kind of task that the preprint is about. And in almost all of the recent cases where deep neural networks have made…

Paper and discussion history. A mention is not an endorsement.

An exact mapping between the Variational Renormalization Group and Deep Learning — mentioned by digital55

Thu, 04 Dec 2014 19:19:33 GMT

This is the paper in question: http://arxiv.org/abs/1410.3831

Paper and discussion history. A mention is not an endorsement.