Do Better ImageNet Models Transfer Better? — mentioned by gwern

Thu, 24 Sep 2020 15:16:35 GMT

Yes, because your validation set performance is imperfectly correlated with performance on more samples (because it's of finite size), so it will tend to overestimate within-distribution. There's also the problem that you probably want to deploy it on data which was not exactly collected the same way. So you have issues of both internal and external validity. In practice, at least for CNN classifiers on ImageNet etc, it seems to be a pretty minor issue overall. Some links: https://arxiv.org/abs/1711.11561 https://arxiv.org/abs/1806.00451 https://arxiv.org/abs/1902.10811…

Paper and discussion history. A mention is not an endorsement.

Do Better ImageNet Models Transfer Better? — mentioned by stared

Thu, 14 Feb 2019 19:39:24 GMT

A few caveats here: - It works (that well) only for vision (for language it sort-of-works only at the word level: http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html) - "Do Better ImageNet Models Transfer Better?" https://arxiv.org/abs/1805.08974 And if you want to play with transfer learning, here is a tutorial with a working notebook: https://deepsense.ai/keras-vs-pytorch-avp-transfer-learning/

Paper and discussion history. A mention is not an endorsement.

Do Better ImageNet Models Transfer Better? · HackerPapers discussions

Do Better ImageNet Models Transfer Better? — mentioned by gwern

Do Better ImageNet Models Transfer Better? — mentioned by stared