Gaussian-Bernoulli RBMs Without Tears · HackerPapers discussions

Gaussian-Bernoulli RBMs Without Tears — mentioned by versteegen

Sat, 12 Oct 2024 15:07:25 GMT

I should mention Restricted Boltzmann Machines (RBMs, which Hinton got the Nobel for): it's really natural to use hidden variables in formulating a problem like this: one lattice of binary variables for GoL time step 0, a second for timestep 1, and optionally a third of continuous variables (or eg 256-level greyscale) for the unquantised target image. (Keep stacking RBMs for more timesteps.) Then you have a bi/tri-partite undirected graphical model. RBMs can be generalised in many ways and one of them is to replace the 2nd-order (pairwise) interactions between variables with higher-order…

Paper and discussion history. A mention is not an endorsement.

Gaussian-Bernoulli RBMs Without Tears — mentioned by versteegen

Tue, 08 Oct 2024 13:14:23 GMT

The landmark Deep Belief Networks (stacked RBMs) paper in Science was in 2006 [1]. DBNs were completely obsolete quite quickly, but don't deny the immense influence of this line of research. It has over 23k citations, and was my introduction to deep learning, for one. And cited by the Nobel committee. You're completely incorrect to say RBMs were of theoretical interest only. They have had plenty of practical use in computer vision/image modelling up to at least a few years ago (I haven't followed them since). Remember the first generative models of human faces? Edit: Wow, Hinton is still…

Paper and discussion history. A mention is not an endorsement.