Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach · HackerPapers discussions

Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach — mentioned by scottmu

Thu, 26 Mar 2026 21:04:24 GMT

I want to clarify what Ken meant by "entropy in the output token probability distributions." Whenever an LLM outputs a token, it's choosing that token out of all possible tokens. Every possible output token has a probability assigned by the model (typically a logarithm of the probability). This is a probability distribution (the output token probabilities sum to 1). Entropy is a measure of uncertainty and can quantify if a token probability distribution is certain (1 token has a 99.9% probability, and the rest share the leftover 0.1% probability) or uncertain (every token has roughly the same…

Paper and discussion history. A mention is not an endorsement.

Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach — mentioned by viraptor

Tue, 08 Oct 2024 13:20:24 GMT

> Doesn't an LLM pick the "most probable next symbol" Yes, but that very rarely matters. (Almost never when it's brought up in discussions) > Couldn't it then, if the probability falls below some threshold, say "I don't know" instead of giving what it knows is a low-probability answer? A low probability doesn't necessarily mean something's incorrect. Responding to your question in French would also have very low probability, even if it's correct. There's also some nuance around what's classified as a hallucination... Maybe something in the training data did suggest that answer as correct.…

Paper and discussion history. A mention is not an endorsement.