Evaluating the World Model Implicit in a Generative Model · HackerPapers discussions

Evaluating the World Model Implicit in a Generative Model — mentioned by godelski

Mon, 20 Oct 2025 04:26:20 GMT

> What if language is the intelligence? Almost certainly not. There does not seem to be a strong correlation between the two. We have a lot of different measures for intelligence when it comes to animals. We can place them across a (multidimensional) spectrum and humans seem unique with language. It also appears that teaching animals language does not cause them to rapidly change on these metrics despite generations of language capabilities. > What if "guessing the next word" really was all that was there, to peak human intelligence, knowledge, and understanding of our world? I believe this…

Paper and discussion history. A mention is not an endorsement.

Evaluating the World Model Implicit in a Generative Model — mentioned by godelski

Sat, 18 Oct 2025 00:05:19 GMT

> any human who ever lived Is this falsifiable? Even restricting to those currently living? On what tests? In which way? Does the category of error matter? > can you name one functional difference between an AI that understands, and one that merely behaves correctly in its domain of expertise? I'd argue you didn't understand the examples from my previous comment or the direct reply[0]. Does it become a duck as soon as you are able to trick an ornithologist? All ornithologists? But yes. Is it fair if I use Go instead of Chess? Game 4 with Lee Sedol seems an appropriate example. Vafa also has…

Paper and discussion history. A mention is not an endorsement.

Evaluating the World Model Implicit in a Generative Model — mentioned by godelski

Mon, 06 Oct 2025 00:59:07 GMT

> Doesn't this then support the claim that LLMs aren't building world models There's actually no strong evidence that LLMs, or any AI system, is actually building a world model. These systems are determined to have "world model" capabilities based on benchmarks, but benchmarks will never be able to tell you if such a feat is taking place. How people are claiming that these have world models is by testing them for consistency. The thing is that a world model is counterfactual. The problems with benchmarks is that they do not distinguish memorization from generalization. To make things worse…

Paper and discussion history. A mention is not an endorsement.

Evaluating the World Model Implicit in a Generative Model — mentioned by godelski

Mon, 04 Aug 2025 17:56:45 GMT

> That's not a contradiction? Correct. No contradiction was intended. As you quote, I wrote "It's not just that". This is not setting up a contrasting point, this is setting up a point that follows. Which, as you point out, does follow. So let me rephrase > If all models are wrong but some are useful then this similarly means that all useful models are wrong in some way. Why flip it around? To highlight the part where they are incorrect as this is what is the thesis of my argument. With that part I do not disagree. > It stands to reason that an ideal next-token predictor would require an…

Paper and discussion history. A mention is not an endorsement.

Evaluating the World Model Implicit in a Generative Model — HN submission

Thu, 07 Nov 2024 05:51:32 GMT

Evaluating the world model implicit in a generative model

159 points; 45 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Evaluating the World Model Implicit in a Generative Model — HN submission

Sat, 22 Jun 2024 09:05:47 GMT

Evaluating the World Model Implicit in a Generative Model

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.