A ConvNet for the 2020s · HackerPapers discussions

A ConvNet for the 2020s — mentioned by sdenton4

Sat, 24 Aug 2024 14:58:54 GMT

Perhaps; in a lot of cases the architecture barely matters. Transformers took a lot of extra tricks to get working well; the ConvNext paper showed that applying those same tricks to convolutional networks can fully close the gap. https://arxiv.org/abs/2201.03545

Paper and discussion history. A mention is not an endorsement.

A ConvNet for the 2020s — mentioned by godelski

Tue, 06 Aug 2024 08:26:48 GMT

> The rule in ML There's definitely attempts to revive things (in the general sense, not just GANs), but most successes appear to come from large labs dedicating equal computational resources to the older models and often by changing names. This can make things more confusing and make things appear to be changing faster, but once you can see this, you'll have an easier time keeping up (so being new, watch out for this). I'll give some examples that are easier to read[0-2] (i.e. don't need expert knowledge to understand the nuances). As an insider (ML researcher), my complaint isn't so much…

Paper and discussion history. A mention is not an endorsement.

A ConvNet for the 2020s — HN submission

Sat, 25 May 2024 03:59:51 GMT

A ConvNet for the 2020s

18 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

A ConvNet for the 2020s — mentioned by michaelt

Wed, 31 Jan 2024 14:00:01 GMT

Academic papers often include an introduction which summarises the context in which the paper is written. Consider https://arxiv.org/pdf/2201.03545v2.pdf where the paper says "For many decades, this has been the default use of ConvNets, generally on limited object categories such as digits [43], faces [58, 76] and pedestrians [19, 63]. Entering the 2010s, the region-based detectors [23, 24, 27, 57] further elevated ConvNets to the position of being the fundamental building block in a visual recognition system." This shows the authors are familiar with the field, avoids inadvertent plagiarism…

Paper and discussion history. A mention is not an endorsement.

A ConvNet for the 2020s — mentioned by nharada

Sun, 09 Apr 2023 18:40:04 GMT

Researchers love the ViT and all its varieties, but a reminder for those following at home that ConvNets still work and scale fine, depending on your requirements. For example, in "A ConvNet for the 2020s"[1], the authors are able to scale up ConvNets to the sizes of ViTs. From the abstract: "Constructed entirely from standard ConvNet modules, ConvNeXts compete favorably with Transformers in terms of accuracy and scalability, achieving 87.8% ImageNet top-1 accuracy and outperforming Swin Transformers on COCO detection and ADE20K segmentation" [1] https://arxiv.org/abs/2201.03545

Paper and discussion history. A mention is not an endorsement.

A ConvNet for the 2020s — mentioned by mrintellectual

Wed, 13 Apr 2022 18:33:22 GMT

> ViT models are outperforming CNNs in terms of computational efficiency and accuracy, achieving highly competitive performance in tasks like image classification, object detection, and semantic image segmentation. Since then, this has been show to be untrue. Using more modern training techniques along with depthwise convolutions (https://arxiv.org/abs/2201.03545) results in equal if not better performance on vision tasks. Improved training methodologies have also been shown to boost the accuracy of ResNet50 - an 6-year-old pure convolutional architecture - on ImageNet-1k by over 5%…

Paper and discussion history. A mention is not an endorsement.

A ConvNet for the 2020s — mentioned by riotnrrd

Wed, 13 Apr 2022 18:05:18 GMT

The performance advantages of transformer architectures have little to do with "transformers" themselves, but rather other architecture changes that were made along the way. They can be matched or beat in performance by tweaked conventional CNNs: https://arxiv.org/abs/2201.03545

Paper and discussion history. A mention is not an endorsement.

A ConvNet for the 2020s — HN submission

Thu, 13 Jan 2022 21:13:00 GMT

A ConvNet for the 2020s

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

A ConvNet for the 2020s — HN submission

Tue, 11 Jan 2022 02:42:10 GMT

A ConvNet for the 2020s

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.