Learning Universal Predictors · HackerPapers discussions

Learning Universal Predictors — mentioned by Xcelerate

Wed, 01 May 2024 17:30:16 GMT

> It is not designed to tell you the "most likely" text that follows, and we don't actually want that. This would mean you have no diversity in your outputs. No, we specifically do want "most likely" to follow; the goal is to approximate Solomonoff induction as well as possible. See this recent paper by Hutter's team: https://arxiv.org/pdf/2401.14953 Quote from the paper: "LLMs pretrained on long-range coherent documents can learn new tasks from a few examples by inferring a shared latent concept. They can do so because in-context learning does implicit Bayesian inference (in line with our…

Paper and discussion history. A mention is not an endorsement.

Learning Universal Predictors — mentioned by tromp

Thu, 29 Feb 2024 07:43:07 GMT

BP is like a slightly messier BrainFuck. Its alphabet size of 17 is rather odd. Its spec (Section E.2 of [1]) says > programs containing unbalanced parentheses are made valid, in particular by skipping any additional ]. But that still leaves programs invalid programs like "[[[[". From a theoretical viewpoint, its biggest flaw is the lack of an input tape, which makes it a non-optimal description method in the sense of [2]. In particular, while BLC programs are at most a constant larger than BP ones, the reverse is not true. The version of BF that I interpret in [3] is still optimally…

Paper and discussion history. A mention is not an endorsement.

Learning Universal Predictors — HN submission

Mon, 29 Jan 2024 05:36:30 GMT

Learning Universal Predictors

66 points; 25 discussion replies on HN. Paper and source history. A submission is not an endorsement.