Why we (usually) don't have to worry about multiple comparisons — mentioned by nextos

Wed, 19 Oct 2022 19:42:53 GMT

A good way to address these issues is to frame all experiments as multilevel models. See [1] for a long discussion from Andrew Gelman et al on why this is advisable. Surprisingly, many people working in statistics still ignore the James-Stein theorem, which provides a theoretical justification for multilevel models. In layman terms, said theorem shows that if you are simultaneously estimating many random variables you should borrow information across variables [2]. Estimating them one by one is suboptimal and does not minimize the global mean squared error. Multilevel models "shrink"…

Paper and discussion history. A mention is not an endorsement.

Why we (usually) don't have to worry about multiple comparisons — mentioned by tristanz

Sun, 23 Feb 2014 22:56:38 GMT

Putting aside bandits and all that, it seems like the first step should be to set up a hierarchical prior which performs shrinkage. Multiple comparisons and stopping issues are largely due to using frequentist tests rather than a simple probabilistic model and inference that conditions and the observed data. Gelman et al, "Why we (usually) don't have to worry about multiple comparisons" http://arxiv.org/abs/0907.2478

Paper and discussion history. A mention is not an endorsement.

Why we (usually) don't have to worry about multiple comparisons · HackerPapers discussions

Why we (usually) don't have to worry about multiple comparisons — mentioned by nextos

Why we (usually) don't have to worry about multiple comparisons — mentioned by tristanz