Why Language Models Hallucinate · HackerPapers discussions

Why Language Models Hallucinate — mentioned by droidjj

Tue, 26 May 2026 20:56:36 GMT

The paper this article is based on: https://arxiv.org/pdf/2509.04664. It's not as sensational as the headline suggests. E.g., "Hallucinations are inevitable only for base models." I feel like this is something most people familiar with how LLMs work already understand.

Paper and discussion history. A mention is not an endorsement.

Why Language Models Hallucinate — HN submission

Fri, 06 Mar 2026 23:21:25 GMT

Why Language Models Hallucinate (2025)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Why Language Models Hallucinate — mentioned by xmcqdpt2

Mon, 02 Feb 2026 11:58:51 GMT

> Honestly the absolute revolution for me would be if someone managed to make LLM tell "sorry I don't know enough about the topic" https://arxiv.org/abs/2509.04664 According to that OpenAI paper, models hallucinate in part because they are optimized on benchmarks that involve guessing. If you make a model that refuses to answer when unsure, you will not get SOTA performance on existing benchmarks and everyone will discount your work. If you create a new benchmark that penalizes guessing, everyone will think you are just creating benchmarks that advantage yourself.

Paper and discussion history. A mention is not an endorsement.

Why Language Models Hallucinate — mentioned by xmcqdpt2

Wed, 10 Dec 2025 06:26:52 GMT

In part because model performance is benchmarked using tests that favor giving partly correct answers as opposed to refusing to answer. If you make a model that doesn't go for part marks, your model will do poorly on all the benchmarks and no one will be interested in it. https://arxiv.org/abs/2509.04664

Paper and discussion history. A mention is not an endorsement.

Why Language Models Hallucinate — mentioned by fragmede

Tue, 14 Oct 2025 15:41:15 GMT

The OpenAI paper on hallucinations gives actual technical reasons for them, if you're interested. https://openai.com/index/why-language-models-hallucinate/ https://arxiv.org/abs/2509.04664

Paper and discussion history. A mention is not an endorsement.

Why Language Models Hallucinate — HN submission

Wed, 24 Sep 2025 20:35:08 GMT

Why Language Models Hallucinate

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Why Language Models Hallucinate — mentioned by janalsncm

Tue, 23 Sep 2025 20:27:28 GMT

> I really don't understand the hallucination problem now in 2025 Perhaps this OpenAI paper would be interesting then (published September 4th): https://arxiv.org/pdf/2509.04664 Hallucination is still absolutely an issue, and it doesn’t go away by reframing it as user error, saying the user didn’t know what they were doing, didn’t know what they needed to get from the LLM, or couldn’t describe it well enough.

Paper and discussion history. A mention is not an endorsement.

Why Language Models Hallucinate — mentioned by nunez

Sat, 13 Sep 2025 18:20:20 GMT

From the abstract of the paper [^0]: > Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty This is a de facto false equivalence for two reasons. First, test takers that are faced with hard questions have the capability of _simply not guessing at all._ UNC did a study on this [^1] by administering a light version of the AMA medical exam to 14 staff members that were NOT trained in the life sciences. While most of the them consistently guessed answers, roughly 6% of them did…

Paper and discussion history. A mention is not an endorsement.

Why Language Models Hallucinate — HN submission

Mon, 08 Sep 2025 15:53:57 GMT

Why Language Models Hallucinate

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.