Neural Kernels Without Tangents — mentioned by vladf

Sat, 16 Apr 2022 15:23:28 GMT

This comment is making a "slight of hand" itself: how do we define the metric for kNN? That will directly impact kNN performance. Using Euclidean norm over pixels on imagenet will not get you anywhere. By recasting classification into a kNN lens you're basically (in the large data regime) recasting the problem to kernel learning under some underspecified RBF kernel with an unknown "proximity" metric instead of Euclidean. In grad school, someone in my lab actually tried scaling kernel methods directly to imagenet too. I don't think that ended up working naively (an interesting neural variant…

Paper and discussion history. A mention is not an endorsement.

Neural Kernels Without Tangents · HackerPapers discussions

Neural Kernels Without Tangents — mentioned by vladf