Compression Represents Intelligence Linearly · HackerPapers discussions

Compression Represents Intelligence Linearly — mentioned by Hasz

Thu, 29 Oct 2026 16:48:15 GMT

I really like the idea that compression is intelligence, there are so many tricks an interesting ideas related to it. A bunch of them are described in the article, but you can imagine a long tail of many more inscrutable tricks chained together to get really incredible shrinkage in data. Hutter prize (I think mentioned in a 3b1b video somewhere else in this thread) is a great example. Given this was written before advent of LLM, I would love to see someone throw openAI/Anthropic levels of compute at a similar problem to see what the swarm can produce. https://arxiv.org/html/2404.09937v1

Paper and discussion history. A mention is not an endorsement.

Compression Represents Intelligence Linearly — mentioned by Legend2440

Sun, 05 May 2024 05:10:03 GMT

LLMs blow away traditional compressors because they are very good predictors, and prediction and compression are the same operation. You can convert any predictor into a lossless compressor by feeding the output probabilities into an entropy coding algorithm. LLMs can get compression ratios as high as 95% (0.4 bits per character) on english text. https://arxiv.org/html/2404.09937v1

Paper and discussion history. A mention is not an endorsement.

Compression Represents Intelligence Linearly — mentioned by jxy

Sun, 05 May 2024 02:28:22 GMT

There was this: Compression Represents Intelligence Linearly, https://arxiv.org/html/2404.09937v1

Paper and discussion history. A mention is not an endorsement.

Compression Represents Intelligence Linearly — HN submission

Tue, 16 Apr 2024 15:10:20 GMT

Compression Represents Intelligence Linearly

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.