HackerPapers

arXiv:2306.03341cs.LG

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model

Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg

Abstract

We introduce Inference-Time Intervention (ITI), a technique designed to enhance the "truthfulness" of large language models (LLMs). ITI operates by shifting model activations during inference, following a set of directions across a limited number of attention heads. This intervention significantly improves the performance of LLaMA models on the TruthfulQA benchmark. On an instruction-finetuned LLaMA called Alpaca, ITI improves its truthfulness from 32.5% to 65.1%. We identify a tradeoff between truthfulness and helpfulness and demonstrate how to balance it by tuning the intervention strength. ITI is minimally invasive and computationally inexpensive. Moreover, the technique is data efficient: while approaches like RLHF require extensive annotations, ITI locates truthful directions using only few hundred examples. Our findings suggest that LLMs may have an internal representation of the likelihood of something being true, even as they produce falsehoods on the surface.

arXiv comments
NeurIPS 2023 spotlight; code: https://github.com/likenneth/honest_llama
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.AIArtificial Intelligence cs.CLComputation and Language

On Hacker News

Linked by 4 identifiable commenters in 4 comments across 4 threads, from Jun 2023 to Aug 2025. Found in 2 HN submissions from Jun 2023 to Jun 2023. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. Persona vectors: Monitoring and controlling character traits in language models 1 comment

    • I was talking to an old colleague/friend about distillation, trying to understand how to steer distillation with regards to removing irrelevant regions of a larger model when training a smaller model. He shared this paper with me, calling the works seminal, it appears to be…

      skhamenehView on HNParent context

  2. ChatGPT Isn't 'Hallucinating'–It's Bullshitting – Scientific American 1 comment

    • > If training data contains multiple conflicting perspectives on a topic, the LLM has a limited ability to recognize that a disagreement is present and what types of entities are more likely to adopt which side. That is what those studies are reflecting. Again, we have empirical…

      agucovaView on HNParent context

  3. The question that no LLM can answer and why it is important 1 comment

    • > demonstrate that the LLM will deny that they know fact X (or be flaky about it, randomly denying and divulging the fact) No, the sampling algorithm you used to query the LLM does that. Not the model itself. e.g. https://arxiv.org/pdf/2306.03341.pdf > B. They know they know…

      astrangeView on HNParent context

  4. Stochastic Parrot 1 comment

    • Yeah. See e.g. https://arxiv.org/abs/2306.03341 > Our findings suggest that LLMs may have an internal representation of the likelihood of something being true, even as they produce falsehoods on the surface. The problem here is that there is currently no reliable way to extract…

      cubefoxView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.