Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo — mentioned by activatedgeek

Mon, 03 Feb 2025 18:09:15 GMT

That has been my understanding too. More generally, a verifier at the end certainly helps. In our paper [1], we find that asking a follow up question like "Is the answer correct?" and taking the normalized probability of "Yes" or "No" token (or more generally any such token trained for) seems to be best bet so far to get well-calibrated probabilities out of the model. In general, the log-probability of tokens is not a good indicator of anything other than satisfying the pre-training loss function of predicting the "next token." (it likely is very well-calibrated on that task though) Semantics…

Paper and discussion history. A mention is not an endorsement.

Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo · HackerPapers discussions

Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo — mentioned by activatedgeek