Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features · HackerPapers discussions

Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features — mentioned by babel_

Thu, 28 Mar 2024 13:59:52 GMT

So, from the perspective I have within the subfield I work in, explainable AI (XAI), we're seeing a bunch of fascinating developments. First, as you mentioned, Rudin continues to prove that the reason for using AI/ML is that we don't understand the problem well enough; otherwise we wouldn't even think to use it! So, pushing our focus to better understand the problem, and then levy ML concepts and techniques (including "classical AI" and statistical learning), we're able to make something that not only outperforms some state-of-the-art in most metrics, but often even is much less resource…

Paper and discussion history. A mention is not an endorsement.

Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features — mentioned by tech_ken

Tue, 19 Mar 2024 18:11:36 GMT

Full pub is up on arXiv here: https://arxiv.org/pdf/2212.13881.pdf My 2c: The phys.org summary isn't great. The authors are focused on a much narrower topic than simply "how NNs learn", they're trying to characterize the mechanism by which deep NNs develop 'features'. They identify a quantity which is definable for each layer of the NN (outer product of the input weight matrix), and posit that this quantity is proportional to the average derivative of the layer with respect to its inputs, where the average is taken over all training data. They (claim to, I haven't evaluated) prove this…

Paper and discussion history. A mention is not an endorsement.

Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features — mentioned by psyklic

Tue, 19 Mar 2024 06:31:35 GMT

Arxiv (2023 paper w/ similar title&authors): https://arxiv.org/abs/2212.13881 Summary by co-author (of the above): https://twitter.com/dbeagleholeCS/status/1627819164906975232

Paper and discussion history. A mention is not an endorsement.