The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets · HackerPapers discussions

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by bakuninsbart

Sun, 03 Aug 2025 19:21:58 GMT

Regarding truth telling, there seems to be some evidence that LLMs at least sometimes "know" when they are lying: https://arxiv.org/abs/2310.06824

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Wed, 30 Oct 2024 15:20:37 GMT

Related: GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 The Internal State of an LLM Knows When It's Lying - https://arxiv.org/abs/2304.13734 LLMs Know More Than What They Say - https://arjunbansal.substack.com/p/llms-know-more-than-what-... Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Tue, 10 Sep 2024 18:53:45 GMT

Pretty much all research (and there's a fair few with different methodologies) on this converge on the same conclusion: LLMs internally know a lot more about the uncertainty and factualness of their predictions than they say. "LLMs are always hallucinating" is a popular stance but wrong all the same. Maybe rather than asking Why models hallucinate, the better question is to ask "Why not?". During pre-training, there's close to zero incentive to push any uncertainty to the forefront (words). GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Sun, 01 Sep 2024 17:07:31 GMT

That's not true. There's been several papers probing this with different methodologies and the conclusion is pretty clear. LLMs know a whole lot more about the uncertainty of their predictions than they say. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824 The Internal State of an LLM Knows When It's Lying - https://arxiv.org/abs/2304.13734 LLMs Know More…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by kromem

Fri, 31 May 2024 23:40:16 GMT

This was AFAIK the first paper to show linear representations of truthiness in LLMs: https://arxiv.org/abs/2310.06824 But what you should really read over is Anthropic's most recent interpretability paper.

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by astrange

Sat, 25 May 2024 22:26:18 GMT

Some things like: https://arxiv.org/abs/2310.18168 https://arxiv.org/abs/2310.06824 There are various reasons an LLM might have incorrect "beliefs" - the input text was false, training doesn't try to preserve true beliefs, quantization certainly doesn't. So it can't be perfectly addressed, but some things leading to it seem like they can be found. > https://arxiv.org/abs/2401.11817 This seems like it's true since LLMs are a finite size, but in Google's case it has a "truth oracle" (the websites it's quoting)… the problem is it's a bad oracle.

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by kromem

Wed, 22 Nov 2023 04:39:30 GMT

> distinguishing fact from fiction Actually this part does seem in recent research to be encoded in LLMs at an abstract level in a linear representation... https://arxiv.org/abs/2310.06824

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Mon, 20 Nov 2023 22:37:23 GMT

>What evidence is there, if any, that LLMs even understand deception as >Deception<? As in, do LLMs understand the concept of Truth, and why other actors might value fidelity to the truth? There is some indication that models internally understand or at least can distinguish truth from falsehood. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in Large Language…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by kromem

Mon, 20 Nov 2023 11:10:25 GMT

There's been a fair bit of research over the last year on this topic. An easy read on a Harvard/MIT study: https://thegradient.pub/othello/ A follow-up on more technical aspects of what's going on with it: https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-o... Two more studies since showing linear representations of world models: https://arxiv.org/abs/2310.02207 (modeling space and time) https://arxiv.org/abs/2310.06824 (modeling truth vs falsehood) It's worth keeping in mind these are all on smaller toy models compared to something like GPT-4, so there's likely more complex versions…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Sat, 18 Nov 2023 20:02:19 GMT

>I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins. Yeah this isn't really true. There's not how humans work. For a variety of reasons, Plenty stick with their incorrect model despite the world indicating otherwise. In fact, this seems to be normal enough human behaviour. Everyone does it, for something or the other. You are no exception. And yes LLMs can in fact tell truth from fiction. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Just Ask for…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Thu, 16 Nov 2023 21:23:49 GMT

>An LLM can't tell the difference between fact and fiction, because it can't apply logic. Not true. They can differentiate it just fine. Of course being able to tell the difference and being incentivized to communicate it are 2 different things. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Mon, 13 Nov 2023 22:03:31 GMT

There really is no difference. Either there is utility or there isn't. "Fake" intelligence that produces results is just something that does not make any sense. LLMs may be worse but Humans confidently state something they are uncertain about all the time lol. Maybe not experts in general but then that's still comparing LLMs to a small percentage of humanity. At any rate, Unlike many seem to think, The issue is not in fact the lack of ability to distinguish truth and fact from fiction. Turns out being able to distinguish the two and having the incentive to communicate that are 2 different…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Sun, 12 Nov 2023 19:05:18 GMT

>The human model isn’t just trained in language though, it’s trained on the world. It's not trained on "the world". It's trained on a small slice of it, a few senses that are themselves slimmed down and then fabricated at parts. To the bird that can feel and sense electromagnetic waves intuitively to guide travels, you're not trained on "the world" >This is why they are so prone to hallucination. No it's not. LLMs can distinguish between truth and hallucination. They just don't care to communicate that. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Just Ask for Calibration…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Wed, 08 Nov 2023 18:32:54 GMT

>More importantly, this is because it doesn't understand the meaning of what it generates, just juggles weights. Yeah that's not the reason at all. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 The Geometry of Truth: Emergent Linear Structure in…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Wed, 08 Nov 2023 04:37:00 GMT

>But the results are indicative of what it is, and they fall short even if your only criteria is usefulness There isn't a testable definition of general intelligence that GPT-4 fails that a chunk of humans also wouldn't. If they fall short then so do people. Guess those people aren't generally intelligent then according to you. >It can't tell the difference between truth and falsity Yes, it very much can. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by visarga

Thu, 19 Oct 2023 19:54:24 GMT

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets > Large Language Models (LLMs) have impressive capabilities, but are also prone to outputting falsehoods. Recent work has developed techniques for inferring whether a LLM is telling the truth by training probes on the LLM's internal activations. However, this line of work is controversial, with some authors pointing out failures of these probes to generalize in basic ways, among other conceptual issues. In this work, we curate high quality datasets of true/false statements and use…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — mentioned by famouswaffles

Fri, 13 Oct 2023 19:30:41 GMT

Language models (the computation) "care" about making "correct" predictions (correct of course being determined by the data). This is why the simplest solution to reducing hallucinations is just making them more competent/better predictors (more neurons, more data). If the prediction can benefit from all the knowledge/reasoning it has accumulated from training, it'll tap that first. One of the biggest tells of hallucinations is if the model is saying something completely different each generation vs being consistent. What they just don't care about is communicating being out of distribution…

Paper and discussion history. A mention is not an endorsement.

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets — HN submission

Thu, 12 Oct 2023 23:50:34 GMT

The Geometry of Truth: Emergent Linear Structure in How LLMs Represent Truth

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.