On the Equivalence Between Deep NADE and Generative Stochastic Networks — mentioned by koningrobot

Mon, 20 Oct 2025 19:38:26 GMT

It goes further back than that. In 2014, Li Yao et al (https://arxiv.org/abs/1409.0585) drew an equivalence between autoregressive (next token prediction, roughly) generative models and generative stochastic networks (denoising autoencoders, the predecessor to difussion models). They argued that the parallel sampling style correctly approximates sequential sampling. In my own work circa 2016 I used this approach in Counterpoint by Convolution (https://arxiv.org/abs/1903.07227), where we in turn argued that despite being an approximation, it leads to better results. Sadly being dressed up as…

Paper and discussion history. A mention is not an endorsement.

On the Equivalence Between Deep NADE and Generative Stochastic Networks · HackerPapers discussions

On the Equivalence Between Deep NADE and Generative Stochastic Networks — mentioned by koningrobot