What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models · HackerPapers discussions

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models — mentioned by ainch

Thu, 11 Jun 2026 18:40:25 GMT

I agree it's an oversimplification. The example I think of is something like Newton's law of gravitation vs Ptolemaic epicycles: one simple explanation replaced many layers of tweaks. It's also a relevant example for AI - one paper tested the ability of Transformers to model planetary orbits: unlike Newton's Law, the implicit forces they learn are nonsense. https://arxiv.org/pdf/2507.06952

Paper and discussion history. A mention is not an endorsement.

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models — mentioned by godelski

Sat, 18 Oct 2025 00:05:19 GMT

> any human who ever lived Is this falsifiable? Even restricting to those currently living? On what tests? In which way? Does the category of error matter? > can you name one functional difference between an AI that understands, and one that merely behaves correctly in its domain of expertise? I'd argue you didn't understand the examples from my previous comment or the direct reply[0]. Does it become a duck as soon as you are able to trick an ornithologist? All ornithologists? But yes. Is it fair if I use Go instead of Chess? Game 4 with Lee Sedol seems an appropriate example. Vafa also has…

Paper and discussion history. A mention is not an endorsement.

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models — mentioned by godelski

Mon, 04 Aug 2025 17:56:45 GMT

> That's not a contradiction? Correct. No contradiction was intended. As you quote, I wrote "It's not just that". This is not setting up a contrasting point, this is setting up a point that follows. Which, as you point out, does follow. So let me rephrase > If all models are wrong but some are useful then this similarly means that all useful models are wrong in some way. Why flip it around? To highlight the part where they are incorrect as this is what is the thesis of my argument. With that part I do not disagree. > It stands to reason that an ideal next-token predictor would require an…

Paper and discussion history. A mention is not an endorsement.

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models — HN submission

Tue, 15 Jul 2025 13:09:20 GMT

LLMs fail to demonstrate internal world model, according to Harvard/MIT study

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models — mentioned by marvin-hansen

Sun, 13 Jul 2025 10:45:40 GMT

Okay, read the abstract and Intro. Recently, in the paper "What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models" your thesis of Ai's lack of capacity to abstract or at least extract understanding from noisy data was largely experimentally confirmed. I am uncertain though about the exact mechanics b/c as they used LLM's, its not transparent what happened internally that lead to constant failure to abstract the concept despite ample predictive power. One interesting experiment was the introduction of the Oracle that literally enabled the LLM to solve the task that…

Paper and discussion history. A mention is not an endorsement.