HackerPapers

arXiv:math/0406077math.ST

A tutorial introduction to the minimum description length principle

Peter Grunwald

Abstract

This tutorial provides an overview of and introduction to Rissanen's Minimum Description Length (MDL) Principle. The first chapter provides a conceptual, entirely non-technical introduction to the subject. It serves as a basis for the technical introduction given in the second chapter, in which all the ideas of the first chapter are made mathematically precise. The main ideas are discussed in great conceptual and technical detail. This tutorial is an extended version of the first two chapters of the collection "Advances in Minimum Description Length: Theory and Application" (edited by P.Grunwald, I.J. Myung and M. Pitt, to be published by the MIT Press, Spring 2005).

arXiv comments
80 pages 5 figures Report with 2 chapters
Journal reference
DOI
Subjects
math.STStatistics Theory cs.ITInformation Theory cs.LGMachine Learning

On Hacker News

Linked by 6 identifiable commenters in 6 comments across 4 threads, from Oct 2013 to Jul 2026.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. 30papers.com – Ilya's 30 essential ML papers, in a beginner friendly format 2 comments

    • https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... https://papers.nips.cc/paper_files/paper/2012/file/c399862d3... https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1511.07122…

      gooobView on HNParent context

    • No need stupid moving texts. CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/ The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/ Understanding LSTM Networks…

      cute_boiView on HNParent context

  2. Text classification with Python 3.14's ZSTD module 1 comment

    • I'm doing my PhD in compression-based machine learning, wanted to contribute a few clarifying points. The relationship between probabilistic modeling and lossless compression is very direct. A model that predicts the next symbol with probability p can on average losslessly…

      jackhurwitzView on HNParent context

  3. Ilya Sutskever: “If you learn all of these, you’ll know 90% of what matters” 2 comments

    • The headline belies the scope of this task. If you click in to see what these are: - one is a full course in convolutional neural networks https://cs231n.github.io/ - one is a 500 page text book https://www.lirmm.fr/~ashen/kolmbook-eng-scan.pdf - another one is an 80 page text…

      klungerView on HNParent context

    • a backup just in case https://nlp.seas.harvard.edu/annotated-transformer/ https://scottaaronson.blog/?p=762 https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://arxiv.org/pdf/1409.2329.pdf…

      is_trueView on HNParent context

  4. Gzip + poetry = awesome 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.