Clustering by compression · HackerPapers discussions

Clustering by compression — mentioned by Kim_Bruning

Thu, 11 Jul 2024 22:25:56 GMT

Thank you for talking with me! I do realize the benefits of the 'compression' model of ML. Sometimes you can even use compression directly, like here: https://arxiv.org/abs/cs/0312044 . I suppose you're right that you only need a few substantively similar outputs to potentially get sued already. (depending on who's scrutinizing you). While talking with you, it occurred to me that so far we've ignored the output set o, which is the set of all images output by -say- stable diffusion. n can then be defined as n = m ∩ o . And we know m is much larger than n, and o is theoretically practically…

Paper and discussion history. A mention is not an endorsement.

Clustering by compression — mentioned by nextos

Thu, 13 Jul 2023 14:43:11 GMT

Something important to note is that the authors use normalized compression distance (NCD). NCD is a way to approximate Kolmogorov complexity. This is a pretty old idea, see [1,2]. Old but still very useful, like perceptrons. [1] Li and Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications. [2] Clustering by compression. https://arxiv.org/pdf/cs/0312044

Paper and discussion history. A mention is not an endorsement.

Clustering by compression — mentioned by sischoel

Tue, 04 May 2021 23:42:52 GMT

This article is surprisingly short on actual applications. For some interesting applications one might look at the the Wikipedia article for the normalized compression distance: https://en.wikipedia.org/wiki/Normalized_compression_distanc... This paper here shows some interesting applications - it is not the first one, but the first non-paywalled one that I found: https://arxiv.org/pdf/cs/0312044.pdf There is also a book by Paul Vitanyi - it has been a while since I looked at it, but if I remember correctly it also discusses some of these applications…

Paper and discussion history. A mention is not an endorsement.

Clustering by compression — mentioned by bediger4000

Sat, 01 Dec 2018 20:48:17 GMT

The author of the article would have been wise to include this information. I did find "Clustering by Compression", https://arxiv.org/abs/cs/0312044 The emphasis in this paper was not on "efficiency", but rather on phylogenetic grouping. The authors used the "Universal Declaration of Human Rights", which apparently has 52 translations as the subject text. See Fig 13 and section 5.2 for details. Instead of just compressing and normalizing, they developed a normalized compression distance, which involves concatenating two texts, compression, and division by the largest of the two texts, as…

Paper and discussion history. A mention is not an endorsement.

Clustering by compression — mentioned by jkabrg

Fri, 10 Aug 2018 22:06:05 GMT

I guess it provides an objective definition of when something is simple or complicated. This could have applications in statistics or machine learning, where "simpler" hypotheses could be less likely to overfit. There are computable measures of "simplicity" (unlike Kolmogorov complexity) based on compression algorithms or computation time (Levin complexity). There are also apparently connections to statistical mechanics, but I haven't looked into it and I don't know if it's useful. See this paper on how to cluster using compression: https://arxiv.org/pdf/cs/0312044.pdf

Paper and discussion history. A mention is not an endorsement.