EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks · HackerPapers discussions

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks — mentioned by sdenton4

Sat, 28 May 2022 16:32:16 GMT

There's a great motivation for small model work for big-model results: More efficient use of compute can be leveraged to make big models effectively bigger. Small-model architectural innovations are computational leverage. You can even see the convolution operation in this light; it's much more efficient than the 'giant dense matrix' approach. EfficientNet is an exemplar of this approach; they made much better small models, and wound up with much higher quality big models as a result of having better architecture overall: https://arxiv.org/pdf/1905.11946.pdf We're currently seeing some great…

Paper and discussion history. A mention is not an endorsement.

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks — mentioned by nl

Fri, 16 Jul 2021 12:21:00 GMT

It's not misleading assuming you know the field. If you don't you might not realize they are comparing ResNet (designed for ultimate performance) vs EffcientNet to show how close it gets in accuracy in the FLOPs budget. Note that the best accuracy for EffcientNet is roughly 1-2% better than ResNet/ResNeXT/SENet/etc but does have a much better FLOPs budget. But these other architectures were never optimised for FLOPs. It wasn't event a consideration when designing them. And EffcientNet is about a (manually designed) technique for scaling neural networks up in accuracy. Only EffcientNet-B0 is…

Paper and discussion history. A mention is not an endorsement.

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks — mentioned by p1esk

Thu, 05 Sep 2019 19:04:02 GMT

the whole idea of "evolving neural networks" didn't pan out To the contrary, evolving neural networks is all the rage these days ("Neural Architecture Search", e.g. SOTA on Imagenet [1]) [1] https://arxiv.org/abs/1905.11946

Paper and discussion history. A mention is not an endorsement.

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks — mentioned by p1esk

Fri, 23 Aug 2019 05:48:47 GMT

Actually this is still very good. Thanks for the links. I'll be timing some of these tricks tomorrow for my Imagenet experiments. By the way, I believe this is the current SOTA for Imagenet: https://arxiv.org/abs/1905.11946 (84/97%). CIFAR10 appears to be essentially solved (99%).

Paper and discussion history. A mention is not an endorsement.

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks — HN submission

Thu, 30 May 2019 13:11:30 GMT

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

119 points; 10 discussion replies on HN. Paper and source history. A submission is not an endorsement.