The Internal State of an LLM Knows When It's Lying · HackerPapers discussions

The Internal State of an LLM Knows When It's Lying — mentioned by pixl97

Fri, 06 Feb 2026 02:40:10 GMT

https://arxiv.org/abs/2304.13734 LLMs have an inner/internal state. Deep neural networks are weird and there is a lot going on in them that makes them very different from the state machines we're used to in binary programs.

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by famouswaffles

Wed, 30 Oct 2024 15:20:37 GMT

Related: GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 The Internal State of an LLM Knows When It's Lying - https://arxiv.org/abs/2304.13734 LLMs Know More Than What They Say - https://arjunbansal.substack.com/p/llms-know-more-than-what-... Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned…

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by famouswaffles

Tue, 10 Sep 2024 18:53:45 GMT

Pretty much all research (and there's a fair few with different methodologies) on this converge on the same conclusion: LLMs internally know a lot more about the uncertainty and factualness of their predictions than they say. "LLMs are always hallucinating" is a popular stance but wrong all the same. Maybe rather than asking Why models hallucinate, the better question is to ask "Why not?". During pre-training, there's close to zero incentive to push any uncertainty to the forefront (words). GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They…

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by Animats

Tue, 10 Sep 2024 08:09:39 GMT

That would be a huge result it if works. The paper: "Our approach is to train a classifier that outputs the probability that a statement is truthful, based on the hidden layer activations of the LLM as it reads or generates the statement. Experiments demonstrate that given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model."[1] It's not entirely clear what they measured and whether what they measured is useful. Is this a major result?…

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by famouswaffles

Sun, 01 Sep 2024 17:07:31 GMT

That's not true. There's been several papers probing this with different methodologies and the conclusion is pretty clear. LLMs know a whole lot more about the uncertainty of their predictions than they say. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 The Internal State of an LLM Knows When It's Lying - https://arxiv.org/abs/2304.13734 LLMs Know More…

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by sanxiyn

Mon, 19 Aug 2024 23:25:13 GMT

I believe the paper to be cited is "The Internal State of an LLM Knows When It's Lying", published last year: https://arxiv.org/abs/2304.13734

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by verdverm

Tue, 21 May 2024 16:12:51 GMT

There are multiple papers and efforts that have inspected the internal state of LLMs. One could even see the word2vec analysis along these lines, as evidence that the model is specializing neurons One such example: The Internal State of an LLM Knows When It's Lying (https://arxiv.org/abs/2304.13734) Searching phrases like "llm interpretability" and "llm activation analysis" uncover more https://github.com/JShollaj/awesome-llm-interpretability

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — HN submission

Mon, 27 Nov 2023 17:56:03 GMT

The Internal State of an LLM Knows When It's Lying

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

The Internal State of an LLM Knows When It's Lying — HN submission

Wed, 27 Sep 2023 12:51:22 GMT

The Internal State of a Large Language Model Knows When Its Lying

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by famouswaffles

Wed, 27 Sep 2023 08:01:49 GMT

The drafts have to be evaluated either by a human or llm. Doing that for every request does not scale when you have millions of users. >Just do the comparison on the user’s machine if the LLM provider is that cheap. This is not possible. Users don't have the resources to run these gigantic models. LLM inference is not cheap. Open ai, Google aren't running profit on free cGPT or Bard. >P.S. Also aren’t LLMs deterministic if you set their “temperature” to zero? Are there drafts if the temperature is zero? If not, then that’s the same as removing the randomness no? It's not a problem of…

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by avereveard

Tue, 01 Aug 2023 09:24:14 GMT

Well it does have concept of right and wrong, as evidenced here https://chat.openai.com/share/e117e800-f579-4eec-9192-e9912f... you can ask it wrong answers only and it will oblige. Correctness is something it learns. I've read a few papers about hallucinations, and the jury is still out on whether a model knows when it's hallucinating, if we assume hallucinations are orthogonal to correctness Now this distinction isn't very useful in the grand scheme of things because in the end the output is wrong anyway, but it doesn't make asking to work along the axis of correctness cargo cult Further…

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — mentioned by T-A

Thu, 01 Jun 2023 22:35:40 GMT

The Internal State of an LLM Knows When its Lying https://arxiv.org/abs/2304.13734

Paper and discussion history. A mention is not an endorsement.

The Internal State of an LLM Knows When It's Lying — HN submission

Mon, 01 May 2023 00:52:53 GMT

The internal state of an LLM knows when it is lying

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.