HackerPapers

arXiv:2104.05874cs.AI

Gradient Kernel Regression

Matt Calder

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

In this article a surprising result is demonstrated using the neural tangent kernel. This kernel is defined as the inner product of the vector of the gradient of an underlying model evaluated at training points. This kernel is used to perform kernel regression. The surprising thing is that the accuracy of that regression is independent of the accuracy of the underlying network.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Oct 2021 to Oct 2021.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. A new link to an old model could crack the mystery of deep learning 1 comment

    • When I first read about the NTK I was motivated to try an experiment to see if the kernel depended on where in the parameter space one evaluated it. I was surprised to see in my simple example the answer was "no". Here is a write up I did: https://arxiv.org/pdf/2104.05874.pdf

      mvcalderView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.