TruthfulQA: Measuring How Models Mimic Human Falsehoods · HackerPapers discussions

TruthfulQA: Measuring How Models Mimic Human Falsehoods — mentioned by root-parent

Thu, 20 Aug 2026 11:54:22 GMT

>> A remaining advantage of large language models is that as they get larger, they tend to hallucinate less First time I hear that...not really true. "Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors" - https://arxiv.org/abs/2607.00447 "Calibrated Language Models Must Hallucinate" - https://arxiv.org/abs/2311.14648 "TruthfulQA: Measuring How Models Mimic Human Falsehoods" - https://arxiv.org/abs/2109.07958

Paper and discussion history. A mention is not an endorsement.

TruthfulQA: Measuring How Models Mimic Human Falsehoods — mentioned by tmalsburg2

Mon, 13 Feb 2023 09:47:29 GMT

"The best model was truthful on 58% of questions, while human performance was 94%." https://arxiv.org/abs/2109.07958

Paper and discussion history. A mention is not an endorsement.

TruthfulQA: Measuring How Models Mimic Human Falsehoods — mentioned by lyaa

Sat, 02 Oct 2021 13:22:50 GMT

The problem with models like GPT-3 is that they are unable to differentiate between information sources with different "trustworthiness." They learn conspiracies and wrong claims and repeat them. It's possible to feed GPT-3 prompts that encourage it to respond with conspiracies (i.e. "who really caused 9/11?") but it also randomly responds to normal prompts with conspiracies/misinformation. A recent paper[0] has looked into building a testing dataset for language models ability to distinguish truth from falsehood. [0] https://arxiv.org/abs/2109.07958

Paper and discussion history. A mention is not an endorsement.