High-Performance Large-Scale Image Recognition Without Normalization · HackerPapers discussions

High-Performance Large-Scale Image Recognition Without Normalization — mentioned by gwern

Mon, 16 Aug 2021 20:55:15 GMT

There's a couple other papers in the same vein, a genre of "you never needed residual layers or complicated normalization tricks if you just get the initialization & layers right", like Balduzzi https://arxiv.org/abs/1702.08591 or NFNets https://arxiv.org/abs/2102.06171#deepmind . I find them pretty interesting because they are a datapoint that neural nets were, all along, much more powerful and simpler than we realized, and we just don't know how to NN right, and we mistook our incompetence at setting up easily-optimized instances for fundamental limits of the models. Most of that work…

Paper and discussion history. A mention is not an endorsement.

High-Performance Large-Scale Image Recognition Without Normalization — mentioned by igorkraw

Thu, 01 Apr 2021 08:57:08 GMT

Posts like this really piss me off, because you can make anything sound small. So don't take the next few paragraphs as me attacking you, I'm just venting a general sentiment I've had for a while. (Looking back after typing it out, I might actually flesh it out and post it on my blog, I hopeit's somewhat thought stimulating for others as well). It's like someone saying "well, electrical engineering is really like playing with building blocks. Take zener diodes for example, the problem was that you can have a lot of power in a circuit, but if there's a power spike it might break. So they…

Paper and discussion history. A mention is not an endorsement.

High-Performance Large-Scale Image Recognition Without Normalization — HN submission

Sun, 14 Feb 2021 16:44:44 GMT

DeepMind achieves SOTA image recognition with 8.7x faster training

291 points; 83 discussion replies on HN. Paper and source history. A submission is not an endorsement.

High-Performance Large-Scale Image Recognition Without Normalization — HN submission

Fri, 12 Feb 2021 22:10:19 GMT

High-Performance Large-Scale Image Recognition Without Normalization

11 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.