Big Transfer (BiT): General Visual Representation Learning — mentioned by gwern

Thu, 24 Sep 2020 15:16:35 GMT

Yes, because your validation set performance is imperfectly correlated with performance on more samples (because it's of finite size), so it will tend to overestimate within-distribution. There's also the problem that you probably want to deploy it on data which was not exactly collected the same way. So you have issues of both internal and external validity. In practice, at least for CNN classifiers on ImageNet etc, it seems to be a pretty minor issue overall. Some links: https://arxiv.org/abs/1711.11561 https://arxiv.org/abs/1806.00451 https://arxiv.org/abs/1902.10811…

Paper and discussion history. A mention is not an endorsement.

Big Transfer (BiT): General Visual Representation Learning — HN submission

Sat, 23 May 2020 17:01:07 GMT

Big Transfer (Bit): General Visual Representation Learning

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Big Transfer (BiT): General Visual Representation Learning · HackerPapers discussions

Big Transfer (BiT): General Visual Representation Learning — mentioned by gwern

Big Transfer (BiT): General Visual Representation Learning — HN submission