Do Language Models Know When They're Hallucinating References? · HackerPapers discussions

Do Language Models Know When They're Hallucinating References? — mentioned by cornel_io

Thu, 20 Mar 2025 05:09:37 GMT

There are various results that suggest that LLMs do internally have everything they'd need to know that they're hallucinating/wrong: https://arxiv.org/abs/2402.09733 https://arxiv.org/abs/2305.18248 https://www.ox.ac.uk/news/2024-06-20-major-research-hallucin... So I don't think it's that they have no concept of correctness, they do, but it's not strong enough. We're probably just not training them in ways that optimize for that over other desirable qualities, at least aggressively enough. It's also clear to anyone who has used many different models over the years that the amount of…

Paper and discussion history. A mention is not an endorsement.

Do Language Models Know When They're Hallucinating References? — mentioned by PoignardAzur

Fri, 06 Oct 2023 21:20:26 GMT

> How do we make sure that the output is factual and not hallucinated? One method the readme doesn't mention: ask the same question multiple times. Apparently research suggests that when LLMs hallucinate answers, their hallucination is likely to be a different one every time, where as factual answers will tend to be consistent. https://arxiv.org/abs/2305.18248

Paper and discussion history. A mention is not an endorsement.

Do Language Models Know When They're Hallucinating References? — mentioned by famouswaffles

Wed, 27 Sep 2023 04:31:06 GMT

https://arxiv.org/abs/2305.18248 "In particular, we find that LMs often hallucinate differing authors of hallucinated references when queried in independent sessions, while consistently identify authors of real references. This suggests that the hallucination may be more a generation issue than inherent to current training techniques or representation." https://arxiv.org/abs/2303.08896 "SelfCheckGPT leverages the simple idea that if a LLM has knowledge of a given concept, sampled responses are likely to be similar and contain consistent facts. However, for hallucinated facts, stochastically…

Paper and discussion history. A mention is not an endorsement.

Do Language Models Know When They're Hallucinating References? — mentioned by avereveard

Tue, 01 Aug 2023 09:24:14 GMT

Well it does have concept of right and wrong, as evidenced here https://chat.openai.com/share/e117e800-f579-4eec-9192-e9912f... you can ask it wrong answers only and it will oblige. Correctness is something it learns. I've read a few papers about hallucinations, and the jury is still out on whether a model knows when it's hallucinating, if we assume hallucinations are orthogonal to correctness Now this distinction isn't very useful in the grand scheme of things because in the end the output is wrong anyway, but it doesn't make asking to work along the axis of correctness cargo cult Further…

Paper and discussion history. A mention is not an endorsement.

Do Language Models Know When They're Hallucinating References? — HN submission

Tue, 30 May 2023 08:34:55 GMT

Do Language Models Know When They're Hallucinating References?

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.