HackerPapers

arXiv:2409.05746stat.ML

LLMs Will Always Hallucinate, and We Need to Live With This

Sourav Banerjee, Ayushi Agarwal, Saloni Singla

Abstract

As Large Language Models become more ubiquitous across domains, it becomes important to examine their inherent limitations critically. This work argues that hallucinations in language models are not just occasional errors but an inevitable feature of these systems. We demonstrate that hallucinations stem from the fundamental mathematical and logical structure of LLMs. It is, therefore, impossible to eliminate them through architectural improvements, dataset enhancements, or fact-checking mechanisms. Our analysis draws on computational theory and Godel's First Incompleteness Theorem, which references the undecidability of problems like the Halting, Emptiness, and Acceptance Problems. We demonstrate that every stage of the LLM process-from training data compilation to fact retrieval, intent classification, and text generation-will have a non-zero probability of producing hallucinations. This work introduces the concept of Structural Hallucination as an intrinsic nature of these systems. By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated.

arXiv comments
Journal reference
DOI
Subjects
stat.MLMachine Learning (Statistics) cs.LGMachine Learning

On Hacker News

Linked by 4 identifiable commenters in 6 comments across 6 threads, from Sep 2024 to Sep 2026. Found in 2 HN submissions from Sep 2024 to Sep 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip 1 comment

    • It is surprising to me that recursive self-improvement seems more plausible now than it did in 2023. Am I the only one to be surprised? I remember the paper proving that hallucinations could never be fully solved back in 2024: https://arxiv.org/abs/2409.05746 I also remember the…

      gozucitoView on HNParent context

  2. O3 Turns Pro 1 comment

    • > Still struggling to find anything in your wall of words that actually backs up your claims. I mean, one easy thing you could do is to explain how LLMs are adequate as >> thinking machines, search engines, knowledge archives, … How much practical experience do you have with…

      ezstView on HNParent context

  3. LLMs know more than they show: On the intrinsic representation of hallucinations 1 comment

    • Much like you can search the internet until you find a source that agrees with you, you can select a set of papers that "confirm" a particular viewpoint, especially in developing fields of research. In this case, the selected papers all support the view LLMs "know what they…

      Retr0idView on HNParent context

  4. Show HN: Last CLI tool you will need 1 comment

    • On what basis you think you will avoid hallucination(s) ? LLM will always hallucinate. People did already some research, among which : https://arxiv.org/abs/2409.05746 and https://arxiv.org/abs/2401.11817 for example.

      az09mugenView on HNParent context

  5. Announcing the Tech for Palestine Incubator 1 comment

    • I'm not overthinking it, some papers describe it [0] and [1] for example. I agree for some subjects you can deal with some error. The problem is not how often but how bad just one single error can be. My point was on controversial topic, where a single error can deal serious…

      az09mugenView on HNParent context

  6. I no longer use AI assistant 1 comment

    • Since the first day I consider AI as a noob working very fast and always making errors. My main usage is for tedious and non-critical tasks such as boilerplates. And I always review what has been generated like a merge/pull request, because in a sense that's what it is. What…

      az09mugenView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.