Notes on a New Philosophy of Empirical Science · HackerPapers discussions

Notes on a New Philosophy of Empirical Science — mentioned by abecedarius

Sun, 09 Feb 2025 17:03:51 GMT

> personally not like to call whatever that is science A counterargument written pre-LLMs: https://arxiv.org/abs/1104.5466

Paper and discussion history. A mention is not an endorsement.

Notes on a New Philosophy of Empirical Science — mentioned by gwern

Wed, 16 Aug 2023 17:36:21 GMT

This is an old idea dating back to the earliest algorithmic information theory work in the '50s/60s like Ray Solomonoff on universal induction: given a powerful learning algorithm, create AI by encoding all possible data as a binary stream (with no further preprocessing or engineering), and simply predict each successive bit. This leads to AIXI, the compression-as-intelligence paradigm (and online learning / meta-learning), and most recently, the success of self-supervised learning & generative modeling. See for example Burfoot https://arxiv.org/abs/1104.5466 or Schmidhuber…

Paper and discussion history. A mention is not an endorsement.

Notes on a New Philosophy of Empirical Science — HN submission

Wed, 12 Jul 2023 18:52:46 GMT

Notes on a New Philosophy of Empirical Science (2011)

38 points; 30 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Notes on a New Philosophy of Empirical Science — mentioned by d_burfoot

Fri, 12 Mar 2021 06:54:19 GMT

I wrote a book about this 10 years ago: https://arxiv.org/abs/1104.5466 Annotation doesn't scale. You are not going to get to real intelligence by labelling billions of data points. I expect that computer vision for autonomous vehicles will become good enough for practical use quite soon after the AV companies start doing large scale self-supervised learning (I don't like that term). Maybe Tesla and Waymo and others are doing that already. Another great application area is medical imaging. Progress in this area has been seriously hampered by the limitations of the annotation paradigm. Once…

Paper and discussion history. A mention is not an endorsement.

Notes on a New Philosophy of Empirical Science — mentioned by d_burfoot

Wed, 13 Sep 2017 16:42:06 GMT

A couple of years ago I wrote a book about a new philosophy of science based on the following methodology: - Obtain a large dataset of observations related to a phenomenon of interest (eg English text, bioimaging data, economic reports, car-mounted camera recordings, etc) - Develop a theory of the phenomenon, or revise a previous theory - Build the theory into a lossless data compression program - Score the theory by invoking the compressor on the dataset, taking into account the size of the compressor itself; lower codelength is better. - Adopt or discard the theory as appropriate and return…

Paper and discussion history. A mention is not an endorsement.

Notes on a New Philosophy of Empirical Science — mentioned by d_burfoot

Tue, 26 Jul 2016 03:54:31 GMT

If you are a hacker who is interested in linguistics, you might want to check out the work we are doing at my startup Ozora Research (http://ozoraresearch.com). Our goal is to build high-quality, sophisticated NLP solutions for "cleantext" : text that has been professionally written and copy-edited, so that it doesn't have obvious errors of spelling, grammar, punctuation, etc. What makes our work different is that, unlike almost all other research in NLP, we don't depend on human-annotated data (like the Penn TreeBank) for training and evaluation. Instead, we build our models into lossless…

Paper and discussion history. A mention is not an endorsement.