HackerPapers

arXiv:1401.7116cs.IT

Bayesian Properties of Normalized Maximum Likelihood and its Fast Computation

Andrew Barron, Teemu Roos, Kazuho Watanabe

Abstract

The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-like representation as a mixture of the component models, even in finite samples, though the weights of linear combination may be both positive and negative. This representation addresses in part the relationship between MDL and Bayes modeling. This representation has the advantage of speeding the calculation of marginals and conditionals required for coding and prediction applications.

arXiv comments
Submitted to ISIT-2004 conference
Journal reference
DOI
Subjects
cs.ITInformation Theory cs.LGMachine Learning stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Aug 2017 to Feb 2020.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Double Slit Experiment and Bayes (2019) 1 comment

    • Negative probabilities interestingly are showing up in various other realms. There seems to be converging interest in the topic from a number of areas. For example, in relating Bayesian inference to algorithmic complexity-based inference: https://arxiv.org/abs/1401.7116 I've…

      skat20physView on HNParent context

  2. The prior can generally only be understood in the context of the likelihood 1 comment

    • A lot of what they discuss is in the literature on reference priors, if not in other literature on objective priors as well. It's a little complex for a comment on HN, but IMHO the best formalization of overfitting is in the literature on minimum description length, and related…

      ta1929901View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.