Reversible Column Networks — mentioned by thomasahle

Fri, 10 Feb 2023 20:46:01 GMT

Nevermind, I guess it's "ImageNet-1K trained models" on which ViT gets 79.9% and the 90% is only when pretraining with ImageNet-22K. There are other non-attention based networks that get 90% too though: https://arxiv.org/pdf/2212.11696v3.pdf

Paper and discussion history. A mention is not an endorsement.

Reversible Column Networks · HackerPapers discussions

Reversible Column Networks — mentioned by thomasahle