Masked Language Model Scoring — mentioned by canjobear

Mon, 20 Jul 2020 04:15:50 GMT

> If you instead produce a list of sentences with similar log probabilities you will see that it contains a mix of grammatical and ungrammatical utterances. Yes, Chomsky mentions this in a footnote. But as far as I know, it hasn't been tried with modern language models. There's been some interesting work that tries to reproduce grammaticality judgments in terms of language model probability after controlling for length and lexical content. It turns out it works pretty well. For instance https://arxiv.org/pdf/1910.14659.pdf

Paper and discussion history. A mention is not an endorsement.

Masked Language Model Scoring · HackerPapers discussions

Masked Language Model Scoring — mentioned by canjobear