How Does Batch Normalization Help Optimization? · HackerPapers discussions

How Does Batch Normalization Help Optimization? — mentioned by D-Machine

Fri, 13 Mar 2026 18:10:04 GMT

Well, one can never be sure what the real motivation for a lot of DL advances, as most papers are post-hoc obscurantism / hand-waving or even just outright nonsense (see: internal covariate shift explanations for batch norm, which arguably couldn't be more wrong https://arxiv.org/pdf/1805.11604). When you really get into this stuff, you tend to see the real motivations as either e.g. kernel smoothing (see comments / discussion at https://news.ycombinator.com/item?id=46357675#46359160) or as encoding correlations / feature similarities / multiplicative interactions (see e.g. broad discussion…

Paper and discussion history. A mention is not an endorsement.

How Does Batch Normalization Help Optimization? — HN submission

Thu, 23 Apr 2020 06:51:23 GMT

How Does Batch Normalization Help Optimization? (2018)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

How Does Batch Normalization Help Optimization? — mentioned by sailingparrot

Sun, 02 Dec 2018 15:32:20 GMT

> What I am reading from you is that basically the only useful part of the ML paper is the abstract. If the repo/blog had a good abstract there would be no point to the paper. The whole point of a blog post is vulgarization (which is definitely needed as well): explaining an idea without assuming that the reader has a lot of prior knowledge in the domain. Which goes in the opposite direction of what researchers look for: the most efficient way to understand what is being proposed. Most of ML papers abstract would be pretty horrible on a blog post in term of vulgarization. Plus, papers are all…

Paper and discussion history. A mention is not an endorsement.