Simple and Effective Masked Diffusion Language Models · HackerPapers discussions

Simple and Effective Masked Diffusion Language Models — mentioned by freeqaz

Thu, 01 May 2025 00:00:57 GMT

Here are two papers linked from Inception's site: 1. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution - https://arxiv.org/abs/2310.16834 2. Simple and Effective Masked Diffusion Language Models - https://arxiv.org/abs/2406.07524

Paper and discussion history. A mention is not an endorsement.

Simple and Effective Masked Diffusion Language Models — mentioned by volodia

Wed, 26 Feb 2025 22:38:52 GMT

Not today, but we will be following up with a technical report over the next week or so. In the meantime, you can take a look at some of the research papers that inspired our work: - https://arxiv.org/abs/2310.16834 - https://arxiv.org/abs/2406.07524

Paper and discussion history. A mention is not an endorsement.

Simple and Effective Masked Diffusion Language Models — mentioned by jxmorris12

Fri, 21 Feb 2025 19:20:45 GMT

Thanks for bringing this up! As far as I understand it current text diffusion models are limited to fairly short context windows. The idea of a text diffusion model continuously updating and revising a million-token-long chain-of-thought is pretty mind-boggling. I agree that these non-autoregressive models could potentially behave in completely different ways. That said, I'm pretty sure we're a long way from building equally-competent diffusion-based base models, let alone reasoning models. If anyone's interested in this topic, here are some more foundational papers to take a look at…

Paper and discussion history. A mention is not an endorsement.