From ImageNet to Image Classification: Contextualizing Progress on Benchmarks — mentioned by logane

Sun, 14 Feb 2021 18:29:13 GMT

Not sure if I follow your JFT argument, but there's a large body of work on both (a) studying whether chasing ImageNet accuracy yields models that generalize well to out of distribution data [1, 2, 3] and (b) contextualizing progress on ImageNet (i.e., what does high accuracy on ImageNet really mean?) [4, 5, 6]. For (a), maybe surprisingly the answer is mostly yes! Better ImageNet accuracy generally corresponds to better out of distribution accuracy. For (b), it turns out that the ImageNet dataset is full of contradictions---many images have multiple ImageNet-relevant objects, and often are…

Paper and discussion history. A mention is not an endorsement.

From ImageNet to Image Classification: Contextualizing Progress on Benchmarks · HackerPapers discussions

From ImageNet to Image Classification: Contextualizing Progress on Benchmarks — mentioned by logane