Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures — mentioned by KhoomeiK

Tue, 08 Dec 2020 21:24:03 GMT

Discovering Symbolic Models from Deep Learning with Inductive Biases [1] trains graph neural nets on astrophysical phenomena and then performs symbolic regression to generate algebraic formulae to elegantly model the phenomena in a classical physics framework. It's largely gone under the radar but has pretty interesting implications for NLP and language theory in my opinion. Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures [2] applies DFA, an approach to training neural nets without backprop, to modern architectures like the Transformer. It does surprisingly…

Paper and discussion history. A mention is not an endorsement.

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures · HackerPapers discussions

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures — mentioned by KhoomeiK