Bad Universal Priors and Notions of Optimality · HackerPapers discussions

Bad Universal Priors and Notions of Optimality — mentioned by chriswarbo

Sun, 24 Nov 2024 21:51:30 GMT

> his AIXI algorithm, which he already considers to be the pinnacle of artificially intelligence This paper from Hutter in 2015 https://arxiv.org/abs/1510.04931 considers AIXI to be "subjective", since the choice of Universal Turing Machine is left unspecified: > We show that Legg-Hutter intelligence and thus balanced Pareto optimality is entirely subjective, and that every policy is Pareto optimal in the class of all computable environments. This undermines all existing optimality properties for AIXI. While it may still serve as a gold standard for AI, our results imply that AIXI is a…

Paper and discussion history. A mention is not an endorsement.

Bad Universal Priors and Notions of Optimality — mentioned by vintermann

Wed, 15 May 2024 10:40:58 GMT

Oops, I thought there was something odd, I got my rationality acronyms mixed up. Hutter's program was called AIXI (MIRI was the research lab). Here is Leike's paper, coauthored with Hutter: https://arxiv.org/abs/1510.04931 They can probably sum it up in their own paper better than I can, but AIXI was supposed to be a formalized, objective model of rationality. They knew from the start that it was uncomputable, but I think they hoped to use it as a sort of gold standard that you could approach. But then it turned out that the choice of Turing machine, which can be (mostly) ignored for…

Paper and discussion history. A mention is not an endorsement.

Bad Universal Priors and Notions of Optimality — mentioned by tarxzvf

Fri, 06 Aug 2021 14:51:09 GMT

What exactly are the right priors for general intelligence? And keep in mind, whichever prior you choose, I can design learning problem where it will lead you astray. This paper provides some interesting results on the weakness inherent in universal priors: https://arxiv.org/abs/1510.04931

Paper and discussion history. A mention is not an endorsement.

Bad Universal Priors and Notions of Optimality — HN submission

Fri, 25 Jun 2021 14:43:39 GMT

Bad Universal Priors and Notions of Optimality (2015)

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.