BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model · HackerPapers discussions

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model — mentioned by boodleboodle

Mon, 21 Sep 2026 04:10:39 GMT

We’ve come full circle to “BERT has a mouth and it must speak” https://arxiv.org/abs/1902.04094

Paper and discussion history. A mention is not an endorsement.

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model — mentioned by thatguysaguy

Mon, 20 Oct 2025 16:21:30 GMT

Back when BERT came out, everyone was trying to get it to generate text. These attempts generally didn't work, here's one for reference though: https://arxiv.org/abs/1902.04094 This doesn't have an explicit diffusion tie in, but Savinov et al. at DeepMind figured out that doing two steps at training time and randomizing the masking probability is enough to get it to work reasonably well.

Paper and discussion history. A mention is not an endorsement.

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model — mentioned by loubbrad

Mon, 20 Oct 2025 15:25:37 GMT

Also relevent - https://arxiv.org/pdf/1902.04094

Paper and discussion history. A mention is not an endorsement.

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model — HN submission

Thu, 21 Feb 2019 05:31:38 GMT

Bert Has a Mouth, and It Must Speak:Bert as a Markov Random Field Language Model

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.