ReZero is All You Need: Fast Convergence at Large Depth · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2003.04887/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTReZero is All You Need: Fast Convergence at Large Depth — mentioned by refibrillatorhttps://news.ycombinator.com/item?id=41685053urn:hackerpapers:mention:2003.04887:41685053Sun, 29 Sep 2024 05:04:56 GMT<p>I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic. Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical and experimental results in ML in the past decade. For example see “dynamical isometry” [1] and the evolution of those ideas which were instrumental in achieving convergence in…</p><p><a href="https://hackerpaper.kaipfister.de/p/2003.04887/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG